# As a condition of accessing this website, you agree to abide by the following # content signals: # (a) If a content-signal = yes, you may collect content for the corresponding # use. # (b) If a content-signal = no, you may not collect content for the # corresponding use. # (c) If the website operator does not include a content signal for a # corresponding use, the website operator neither grants nor restricts # permission via content signal with respect to the corresponding use. # The content signals and their meanings are: # search: building a search index and providing search results (e.g., returning # hyperlinks and short excerpts from your website's contents). Search does not # include providing AI-generated search summaries. # ai-input: inputting content into one or more AI models (e.g., retrieval # augmented generation, grounding, or other real-time taking of content for # generative AI search answers). # ai-train: training or fine-tuning AI models. # ANY RESTRICTIONS EXPRESSED VIA CONTENT SIGNALS ARE EXPRESS RESERVATIONS OF # RIGHTS UNDER ARTICLE 4 OF THE EUROPEAN UNION DIRECTIVE 2019/790 ON COPYRIGHT # AND RELATED RIGHTS IN THE DIGITAL SINGLE MARKET. # BEGIN Cloudflare Managed content User-Agent: * Content-signal: search=yes,ai-train=no Allow: / User-agent: Amazonbot Disallow: / User-agent: Applebot-Extended Disallow: / User-agent: Bytespider Disallow: / User-agent: CCBot Disallow: / User-agent: ClaudeBot Disallow: / User-agent: Google-Extended Disallow: / User-agent: GPTBot Disallow: / User-agent: meta-externalagent Disallow: / # END Cloudflare Managed Content # GENERAL NOTICE TO ALL BOTS # # If for some strange reason your bot wants to login and do things, that's cool. # However, your bot MUST respect session cookies like a normal browser. Else, I will prevent your bot from getting a cookie. # # Due to these user agents below not following standard cookies, the following user agents are prevented from getting a session, therefore a cookie, and therefore able to login. # This is not a punishment so much as just prevent clogging the session store of you search engines that don't care about cookies, so why waste the space? :) # # 'msnbot', # Microsoft Bing search engine # 'yahoo! slurp', # Yahoo! search engine # 'googlebot', # Google search engine # 'mediapartners', # Google AdSense spider # 'feedfetcher-google', # Google Feedfetcher RSS fetcher # 'teoma', # Ask Jeeves search engine # 'wordpress', # Wordpress pingbacks # 'baiduspider', # Chinese search/MP3 engine # 'sparkflare', # Sparkflare feed fetcher for Campfire # 'bingbot', # Bing changed # 'downscout', # 'ltx71', # 'magpie-crawler', #Brandwatch? Dunno but they are MASSIVELY annoying and 100 times bigger than the next bot # 'ahrefsbot', # 'oauth', # '360spider', # 'psbot' # # See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file # # To ban all spiders from the entire site uncomment the next two lines: # User-Agent: * # Disallow: / User-Agent: * Disallow: /forum/Secret/ User-Agent: * Disallow: /login/ User-Agent: * Disallow: /oauth/ User-Agent: MJ12bot Disallow: / User-agent: CCBot Disallow: / User-agent: ChatGPT-User Disallow: / User-agent: GPTBot Disallow: / User-agent: Google-Extended Disallow: / User-agent: anthropic-ai Disallow: / User-agent: Omgilibot Disallow: / User-agent: Omgili Disallow: / User-agent: FacebookBot Disallow: / User-Agent: ImagesiftBot Disallow: / User-Agent: meta-externalagent Disallow: /