User-agent: *Disallow: /PHPTesting/Disallow: /_private/Disallow: /cgi-bin/Disallow: /images/Disallow: /Pictures_Darren/Disallow: /scgi-bin/Disallow: /sounds/Disallow: /BackupPages/Disallow: /Download_Files/Disallow: /DarrenEdit/
Disallow: /HVMenu/# don't let search engines see the RDF feed, it's just confusing.User-agent: Googlebot/2.1User-agent: InfoNaviRobot(F107)User-agent: TV33_Mercator_1-1.0User-agent: AVSearch-3.0User-agent: Scooter/2.0User-agent: Slurp/2.0User-agent: SearchengineLicenceSheep_v1.0User-agent: shadow/2.0User-agent: MultiText/0.1User-agent: FAST-WebCrawler/2.2.5User-agent: Atomz/1.0User-agent: htdig/ (searchit@netmind.com)User-agent: spider00.logika.net.# updated 2002-6-2