# Tells Scanning Robots Where They Are And Are Not Welcome # User-agent: can also specify by name; "*" is for everyone # Disallow: if this matches first part of requested path, # forget it User-agent: twiceler Disallow: / User-agent: Baiduspider Disallow: / User-agent: * # applies to all robots Disallow: /mail Disallow: /images Disallow: /Images Disallow: /stats Disallow: /icons Disallow: /swn_maps Disallow: /SWN-icons Disallow: /arrows Disallow: /backup Disallow: /marketingontap