# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file # # To ban all spiders from the entire site uncomment the next two lines: # User-Agent: * # Disallow: / User-agent: Googlebot Disallow: User-agent: Googlebot-image Disallow: User-agent: * Sitemap: https://mj-spree-production.s3.amazonaws.com/sitemaps/sitemap.xml.gz Disallow: /checkouts Disallow: /orders Disallow: /countries Disallow: /line_items Disallow: /password_resets Disallow: /user_sessions Disallow: /users Disallow: /shop/cart Disallow: /shop/checkout Disallow: /shop/checkout/address Disallow: /shop/checkout/delivery Disallow: /shop/states Disallow: /community/posts/new Disallow: /static/modal Disallow: /profile User-agent: NinjaBot Allow: /