# /robots.txt 

User-agent: *
crawl-delay: 5
Disallow: /aabot1/
Allow: /aabot2/