# robots.txt User-agent: Yahoo! Slurp User-agent: Slurp User-agent: Googlebot User-agent: Googlebot-Image User-agent: AbachoBOT User-agent: ArchitextSpider User-agent: Arachnoidea User-agent: Atomz User-agent: AxmoRobot User-agent: Clushbot User-agent: EZResult User-agent: FAST-WebCrawler User-agent: Fluffy the spider User-agent: ia_archiver User-agent: MantraAgent User-agent: MSN User-agent: MSNBot User-agent: Scooter User-agent: Mercator User-agent: Speedy Spider User-agent: Teoma User-agent: Gigabot User-agent: Scrubby User-agent: Robozilla User-agent: Nutch User-agent: baiduspider User-agent: yahoo-mmcrawler User-agent: psbot User-agent: asterias User-agent: yahoo-blogs/v3.9 User-agent: Winona User-agent: WebCrawler User-agent: Webwombat User-agent: ZyBorg Disallow: User-agent: * Disallow: /cgi-bin/ Disallow: /scripts/