=================================================================== Zcool(www.zcool.com.cn) robots.txt Last updated: 2025-10-21 This file provides directives to search engine crawlers and other bots. Our goal is to ensure efficient indexing of our high-quality creative assets while protecting our crawl budget and preventing scraper abuse. =================================================================== --- 核心指令：Sitemap 索引文件入口 --- 这是给所有搜索引擎的最重要指令，指明了我们所有内容的地图。 Sitemap:https://www.zcool.com.cn/sitemap.txt Sitemap:https://www.zcool.com.cn/sitemap_work_99.txt Sitemap:https://www.zcool.com.cn/sitemap_tag_73.txt Sitemap:https://www.zcool.com.cn/sitemap_assets_235.txt Sitemap:https://www.zcool.com.cn/sitemap_assets_tag_73.txt =================================================================== Section 1: 对所有主流搜索引擎和AI爬虫的通用规则 我们采用“默认允许，按需禁止”的策略。 适用于 Googlebot, Bingbot, Baiduspider, Sogou Spider, 360Spider, YandexBot, DuckDuckBot, Google-Extended, ChatGPT-User, PerplexityBot 等。 =================================================================== User-agent: * --- 禁止抓取的部分 --- 1. 后台、管理和内部系统路径 Disallow: /admin/ Disallow: /api/ Disallow: /login Disallow: /logout 2. 用户个人中心、购物车和订单流程 Disallow: /u/*/profile Disallow: /messages/ Disallow: /setting 3. 站内搜索结果页 防止搜索引擎抓取无限的、低质量的搜索结果组合。 Disallow: /search/ Disallow: /search 4. 参数化URL、筛选、排序和分页 Disallow: /*?* # 5. 禁止访问特定的文件类型 Disallow: /*.php$ Disallow: /*.inc$ Disallow: /*.cgi$ Disallow: /*.pl$ Disallow: /*.zip$ Disallow: /*.rar$ Disallow: /*.pdf$ --- 明确允许的部分 --- Allow:/tag/ Allow:/work/ Allow:/u/ Allow:/article/ Allow:/assets/ Allow:/collection/ Allow:/retouching/ Allow:/gfx/ Allow:/event/ Allow:/about/ Allow:/subscription/ Allow:/activity/ Allow:/top/ Allow:/special/ Allow:/specials/ Allow:/workimage/ Allow:/tag Allow:/work Allow:/u Allow:/article Allow:/assets Allow:/collection Allow:/retouching Allow:/gfx Allow:/event Allow:/about Allow:/subscription Allow:/activity Allow:/top Allow:/special Allow:/specials Allow:/workimage ---明确不允许的部分--- User-agent: Googlebot-Image Disallow: /search/ Disallow: /u/*/profile Disallow: /login Disallow: /logout Disallow: /messages/ Disallow: /setting User-agent: Mediapartners-Google Disallow: /search/ Disallow: /u/*/profile Disallow: /login Disallow: /logout Disallow: /messages/ Disallow: /setting User-agent: Baiduspider Disallow: /search/ Disallow: /u/*/profile Disallow: /login Disallow: /logout Disallow: /messages/ Disallow: /setting User-agent: Baiduspider-news Disallow: /search/ Disallow: /u/*/profile Disallow: /login Disallow: /logout Disallow: /messages/ Disallow: /setting User-agent: Baiduspider-render Disallow: /search/ Disallow: /u/*/profile Disallow: /login Disallow: /logout Disallow: /messages/ Disallow: /setting User-agent: Baiduspider-image Disallow: /search/ Disallow: /u/*/profile Disallow: /login Disallow: /logout Disallow: /messages/ Disallow: /setting User-agent: Sogou Disallow: /search/ Disallow: /u/*/profile Disallow: /login Disallow: /logout Disallow: /messages/ Disallow: /setting User-agent: Yahoo! Slurp Disallow: /search/ Disallow: /u/*/profile Disallow: /login Disallow: /logout Disallow: /messages/ Disallow: /setting User-agent: bingbot Disallow: /search/ Disallow: /u/*/profile Disallow: /login Disallow: /logout Disallow: /messages/ Disallow: /setting User-agent: 360Spider Disallow: /search/ Disallow: /u/*/profile Disallow: /login Disallow: /logout Disallow: /messages/ Disallow: /setting User-agent: HaosouSpider Disallow: /search/ Disallow: /u/*/profile Disallow: /login Disallow: /logout Disallow: /messages/ Disallow: /setting User-agent: yisouspider Disallow: /search/ Disallow: /u/*/profile Disallow: /login Disallow: /logout Disallow: /messages/ Disallow: /setting User-agent: YoudaoBot Disallow: /search/ Disallow: /u/*/profile Disallow: /login Disallow: /logout Disallow: /messages/ Disallow: /setting User-agent: Sogou Orion spider Disallow: /search/ Disallow: /u/*/profile Disallow: /login Disallow: /logout Disallow: /messages/ Disallow: /setting User-agent: Sogou News Spider Disallow: /search/ Disallow: /u/*/profile Disallow: /login Disallow: /logout Disallow: /messages/ Disallow: /setting User-agent: Sogou blog Disallow: /search/ Disallow: /u/*/profile Disallow: /login Disallow: /logout Disallow: /messages/ Disallow: /setting User-agent: Sogou spider2 Disallow: /search/ Disallow: /u/*/profile Disallow: /login Disallow: /logout Disallow: /messages/ Disallow: /setting User-agent: Sogou web spider Disallow: /search/ Disallow: /u/*/profile Disallow: /login Disallow: /logout Disallow: /messages/ Disallow: /setting User-agent: EasouSpider Disallow: /search/ Disallow: /u/*/profile Disallow: /login Disallow: /logout Disallow: /messages/ Disallow: /setting User-agent: MSNBot Disallow: /search/ Disallow: /u/*/profile Disallow: /login Disallow: /logout Disallow: /messages/ Disallow: /setting User-agent: Yandex Disallow: /search/ Disallow: /u/*/profile Disallow: /login Disallow: /logout Disallow: /messages/ Disallow: /setting User-agent: ia_archiver Disallow: /search/ Disallow: /u/*/profile Disallow: /login Disallow: /logout Disallow: /messages/ Disallow: /setting User-agent: IPS-Agent Disallow: /search/ Disallow: /u/*/profile Disallow: /login Disallow: /logout Disallow: /messages/ Disallow: /setting User-agent: BLEXBot Disallow: /search/ Disallow: /u/*/profile Disallow: /login Disallow: /logout Disallow: /messages/ Disallow: /setting =================================================================== Section 2: 防御性规则 - 限制已知的非必要爬虫和恶意爬虫 robots.txt 是一个“君子协定”。流氓爬虫和恶意脚本会完全忽略这些规则 这部分是第一道脆弱的防线。 =================================================================== # --- 屏蔽一些对 SEO 无益的 SEO 工具爬虫 --- User-agent: AhrefsBot Disallow: / User-agent: SemrushBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: DotBot Disallow: / User-agent: BLEXBot Disallow: / User-agent: AhrefsBot Disallow: / --- 对一些行为尚可但过于频繁的爬虫设置抓取延迟 --- 注意：Googlebot 、百度、搜搜不遵守此指令，但 Bing, Yandex 等会遵守。 设置一个延迟可以减轻服务器压力。单位：秒。 User-agent: Bytespider Crawl-delay: 5 User-agent: PetalBot Crawl-delay: 5 User-agent: bingbot Crawl-delay: 5 User-agent: YandexBot Crawl-delay: 5 User-agent: Slurp Crawl-delay: 5 =================================================================== Section 3: 针对特定图片、视频爬虫的规则 (可选，通常*规则已足够) # 确保图片爬虫可以无障碍地访问所有内容。 #=================================================================== User-agent: Googlebot-Image Allow:/tag/ Allow:/work/ Allow:/u/ Allow:/article/ Allow:/assets/ Allow:/collection/ Allow:/retouching/ Allow:/gfx/ Allow:/event/ Allow:/about/ Allow:/subscription/ Allow:/activity/ Allow:/top/ Allow:/special/ Allow:/specials/ Allow:/workimage/ Allow:/tag Allow:/work Allow:/u Allow:/article Allow:/assets Allow:/collection Allow:/retouching Allow:/gfx Allow:/event Allow:/about Allow:/subscription Allow:/activity Allow:/top Allow:/special Allow:/specials Allow:/workimage User-agent: Baiduspider-Image Allow:/tag/ Allow:/work/ Allow:/u/ Allow:/article/ Allow:/assets/ Allow:/collection/ Allow:/retouching/ Allow:/gfx/ Allow:/event/ Allow:/about/ Allow:/subscription/ Allow:/activity/ Allow:/top/ Allow:/special/ Allow:/specials/ Allow:/workimage/ Allow:/tag Allow:/work Allow:/u Allow:/article Allow:/assets Allow:/collection Allow:/retouching Allow:/gfx Allow:/event Allow:/about Allow:/subscription Allow:/activity Allow:/top Allow:/special Allow:/specials Allow:/workimage User-agent: BingPreview Allow:/tag/ Allow:/work/ Allow:/u/ Allow:/article/ Allow:/assets/ Allow:/collection/ Allow:/retouching/ Allow:/gfx/ Allow:/event/ Allow:/about/ Allow:/subscription/ Allow:/activity/ Allow:/top/ Allow:/special/ Allow:/specials/ Allow:/workimage/ Allow:/tag Allow:/work Allow:/u Allow:/article Allow:/assets Allow:/collection Allow:/retouching Allow:/gfx Allow:/event Allow:/about Allow:/subscription Allow:/activity Allow:/top Allow:/special Allow:/specials Allow:/workimage User-agent: Pinterestbot Allow:/tag/ Allow:/work/ Allow:/u/ Allow:/article/ Allow:/assets/ Allow:/collection/ Allow:/retouching/ Allow:/gfx/ Allow:/event/ Allow:/about/ Allow:/subscription/ Allow:/activity/ Allow:/top/ Allow:/special/ Allow:/specials/ Allow:/workimage/ Allow:/tag Allow:/work Allow:/u Allow:/article Allow:/assets Allow:/collection Allow:/retouching Allow:/gfx Allow:/event Allow:/about Allow:/subscription Allow:/activity Allow:/top Allow:/special Allow:/specials Allow:/workimage User-agent: Googlebot-Video Allow:/tag/ Allow:/work/ Allow:/u/ Allow:/article/ Allow:/assets/ Allow:/collection/ Allow:/retouching/ Allow:/gfx/ Allow:/event/ Allow:/about/ Allow:/subscription/ Allow:/activity/ Allow:/top/ Allow:/special/ Allow:/specials/ Allow:/workimage/ Allow:/tag Allow:/work Allow:/u Allow:/article Allow:/assets Allow:/collection Allow:/retouching Allow:/gfx Allow:/event Allow:/about Allow:/subscription Allow:/activity Allow:/top Allow:/special Allow:/specials Allow:/workimage User-agent: Baiduspider-image Allow:/tag/ Allow:/work/ Allow:/u/ Allow:/article/ Allow:/assets/ Allow:/collection/ Allow:/retouching/ Allow:/gfx/ Allow:/event/ Allow:/about/ Allow:/subscription/ Allow:/activity/ Allow:/top/ Allow:/special/ Allow:/specials/ Allow:/workimage/ Allow:/tag Allow:/work Allow:/u Allow:/article Allow:/assets Allow:/collection Allow:/retouching Allow:/gfx Allow:/event Allow:/about Allow:/subscription Allow:/activity Allow:/top Allow:/special Allow:/specials Allow:/workimage # =================================================================== # Section 4: AI 模型训练爬虫规则 (保护策略) # # 我们欢迎AI用于搜索发现，但禁止将我们的版权内容用于模型训练。暂时禁止CCBot、AppleBot、Alexabot对SEO无较大影响 # These directives are intended to opt out of training for generative AI models. # =================================================================== User-agent: GPTBot Allow: / User-agent: ClaudeBot Allow: / User-agent: PerplexityBot Allow: / User-agent: YisouSpider Allow: / User-agent: DeepseekSpider Allow: / User-agent: CCBot Disallow: / User-agent: Applebot # 苹果的规则比较模糊，保守起见可以先禁止 Disallow: / User-agent: Alexabot Disallow: /