Robots exclusion rules
Discovery
Robots exclusion rules
Standard robots.txt at the site root telling crawlers which paths are allowed/disallowed and pointing at the sitemap.
GET
Robots exclusion rules
Response
200 - text/plain
200 response
The response is of type string.