# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file # # To ban all spiders from the entire site uncomment the next two lines: # User-agent: * # Disallow: / User-agent: EasouSpider Disallow: / User-agent: MJ12bot Disallow: / User-agent: XoviBot Disallow: / User-agent: AhrefsBot Disallow: / User-agent: * Disallow: /ping User-agent: * Disallow: /api User-agent: bitlybot Disallow: / User-agent: * Disallow: /lib/pdf_forms/metadata/ User-agent: * Disallow: /lacountyassist Sitemap: https://s3-.amazonaws.com//production/sitemaps/sitemap.xml.gz