Robots.txt change monitoring

This weekend we launched Robots.txt monitoring. We check every day the contents of the robots.txt and let you know if it changes, what was added and what was removed.

Nothing much else to say really?

Leave a Reply

Your email address will not be published. Required fields are marked *