User-agent: * Allow: / Disallow: /*?q= Sitemap: https://51crdj.com/sitemap.xml Host: https://51crdj.com User-agent: Baiduspider Allow: / Disallow: /*?q= Crawl-delay: 1 Sitemap: https://51crdj.com/sitemap.xml User-agent: Sogou web spider Allow: / Disallow: /*?q= Sitemap: https://51crdj.com/sitemap.xml User-agent: 360Spider Allow: / Disallow: /*?q= Sitemap: https://51crdj.com/sitemap.xml User-agent: HaosouSpider Allow: / Disallow: /*?q= Sitemap: https://51crdj.com/sitemap.xml User-agent: Bingbot Allow: / Disallow: /*?q= Sitemap: https://51crdj.com/sitemap.xml User-agent: Googlebot Allow: / Disallow: /*?q= Sitemap: https://51crdj.com/sitemap.xml User-agent: Googlebot-Image Allow: /image/ Sitemap: https://51crdj.com/sitemap.xml User-agent: YandexBot Allow: / Disallow: /*?q= Crawl-delay: 1 Sitemap: https://51crdj.com/sitemap.xml User-agent: YandexImages Allow: /image/ Sitemap: https://51crdj.com/sitemap.xml User-agent: AhrefsBot Disallow: / User-agent: SemrushBot Disallow: / User-agent: MJ12bot Disallow: /