# Robots.txt file from http://www.meishanxiu.com # All robots will spider the domain User-Agent: * Disallow:/coll/member Disallow:/coll/webvisit User-agent: twiceler Disallow: / # User-agent: TurnitinBot Disallow: / #Mozilla/5.0 (compatible; MJ12bot/v1.2.5; http://www.majestic12.co.uk/bot.php?+) User-agent: MJ12bot Disallow: /