Say hello to X-Robots Header and Meta Robots Scanning

That’s right, two of the most requested features since our initial launch is the ability to scan for both the X-Robots HTTP Header and Meta Tag. 

The custom configuration is the same at Robots.txt indexability, you can specify if something should be indexable or not (the default is Indexable but if you have a staging or development site, you can easily switch this.

You can activate these right now on your URLs by flipping the switch.

Leave a Reply

Your email address will not be published. Required fields are marked *