Crawl Delay
You can use the crawl delay option in your robots.txt file to specify how often our robots will retrieve pages from your website.
The following examples will set a crawl delay of 30 seconds. The crawl delay must be in seconds.
User-Agent: *
Crawl-Delay: 30
Or specific to startShoppingBot:
User-Agent: startShoppingBot
Crawl-Delay: 30