I regularly use a couple of AI resources to create content — not so much blue-sky generative efforts as much as polishing up my own writing or content supplied by clients. However, recently I’ve found myself being protective about LLM/AI appropriation of specific content on specific client sites. Opting in to using ChatGPT for my own purposes shouldn’t be a tacit endorsement that my clients have to abide by.
For whatever reason, I’d made out the process to be a lot more labor-intensive and baroque than it really is. All you need to do is create or update your robots.txt file with the following:
User-agent: GPTBot
Disallow: /
User-agent: ChatGPT-User
Disallow: /
User-agent: Google-Extended
Disallow: /
User-agent: GPTBot
Disallow: /
User-agent: CCBot
Disallow: /
User-agent: ChatGPT-User
Disallow: /
User-agent: Google-Extended
Disallow: /
User-agent: Omgilibot
Disallow: /
User-Agent: FacebookBot
Disallow: /
You can download a turnkey .txt file of the above here.
Microsoft Bing
Bing requires a meta tag instead of respecting robots.txt
<meta name="bingbot" content="nocache">
Shout out to the following for the quick education in LLM blocking:
![](https://www.joeyoungblood.com/wp-content/uploads/2022/12/openai-chatgpt-logo.png)
![](https://www.coywolf.news/wp-content/uploads/2023/09/og-google-extended.webp)
![](https://www.privacyjournal.net/wp-content/uploads/2023/10/Protect-Your-Site-From-ChatGPT-How-to-Block-LLM-Crawlers-PJ.png)
![](https://www.maginative.com/content/images/2023/09/bing-chat-robots.jpg)
![](https://yoast.com/app/uploads/2022/02/help_image_og.png)
Leave a Reply