# This is a file retrieved by spiders/robots that conform to a # defacto standard. See: # # # User-agent: * Disallow: /cgi-bin User-agent: LinkWalker Disallow: / User-agent: TurnitinBot Disallow: / User-Agent: sitecheck.internetseer.com Disallow: / User-Agent: obot Disallow: /