# All robots will spider the domain User-agent: * Disallow: /admin/ Disallow: /customer/ Disallow: /database/ Disallow: /files/ Disallow: /includes/ Disallow: /log/ Disallow: /mail/ Disallow: /modules/ Disallow: /payment/ Disallow: /schemes/ Disallow: /shipping/ Disallow: /Smarty-2.6.12/ Disallow: /sns_collector/ Disallow: /skin1_original/ Disallow: /skin1/ Disallow: /sql/ Disallow: /templates_c/ Disallow: /upgrade/