Search Appliance
The Search Appliance comes with no robots.txt file. If you want to control
how web walking robots visit your search pages you can upload a
robots.txt file. This robots.txt file will apply to all pages
returned bythe Search Appliance including admin, documentation, and searches
whether accessed by HTTP or HTTPS.
See http://www.robotstxt.org for the syntax of robots.txt files.
If you no longer wish to have a robots.txt check Delete.
Note: You may also control robots for searches on a profile by profile basis
by using meta robots within the top HTML of the search settings or the
custom XSL if you're using an XSL style sheet for search results.
See http://www.robotstxt.org for the syntax of the meta robots tag.