Google, Microsoft & OpenAI Plan To Honor Shared AI Training Opt-Out
The new policy token lets websites block generative AI training with one robots.txt rule, and Microsoft says it will honor it in 2027.
- Google, Microsoft, OpenAI, and Anthropic announced Friday support for a shared robots.txt standard allowing publishers to collectively opt out of AI training with a single rule.
- Publishers have increasingly challenged the use of their content for AI training without permission or compensation. Previously, site owners had to write separate rules for each company, a process critics say is cumbersome.
- Kevin Kelley, a technical lead for standards at Anthropic, proposed the standard through an Internet Engineering Task Force draft published Friday. Sites add a shared token called "__usage_ai-training" to robots.txt to block multiple AI companies from training on their content.
- Anthropic has already begun adopting the signal, while OpenAI plans to implement it in the coming months. Google and Microsoft intend to support the standard next year, with Microsoft starting February 1, 2027.
- Microsoft will replace its older NoArchive and NoCache tags with the new preferences starting February 1, 2027. Sites using the legacy tags must switch by March 1, 2027, or lose crawler preference recognition.
23 Articles
23 Articles
OpenAI, Google, and Microsoft Introduce Integrated Standard to Reject AI Training on Website Content
Major artificial intelligence (AI) companies, including OpenAI, Google, and Microsoft (MS), announced on the 31st (local time) that they have agreed to support a common standard that allows website operators to block the collection of AI training data all at once. Until now, operators had to block crawling bots for each company one by one in a ‘robots.txt’ file and update it whenever a new bot appeared. Once the standard is applied, ‘AI training…
AI training opt-out: publishers ‘should not have to choose’
Google, Microsoft, OpenAI and Anthropic will honour one shared AI training opt-out in robots.txt. It lets a website keep its content out of generative AI training with a single rule. Today, a site that wants to say no has to write separate rules for each company’s crawler. The shared signal comes from an IETF draft […] This story continues at The Next Web
New Policy Token Could Give Publishers One Shared Way to Reject AI-Training Crawls
A proposed policy token could give publishers one shared way to tell participating crawlers that they do not want their website content used for model training. This post: New Policy Token Could Give Publishers One Shared Way to Reject AI-Training Crawls is first published on Compsmag.com
Coverage Details
Bias Distribution
- 50% of the sources lean Left
Factuality
To view factuality data please Upgrade to Premium















