Published • loading... • Updated
AI giants promise publishers new way to block some scraping
- On Friday, OpenAI, Google, Microsoft, and Anthropic pledged to give websites a single method to prevent their content from being scraped to train AI models, with companies restricting crawlers that encounter the shared token.
- Publishers currently manage numerous blocking commands for every individual bot crawling the web, a burden the new protocol eliminates by allowing them to apply AI training preferences across multiple companies simultaneously.
- Kevin Kelley, a technical lead at Anthropic, proposed the shared "Policy token." signal for the Internet Engineering Task Force; Meta spokesman Andy Stone conveyed support, encouraging cross-industry adoption.
- Claude-Maker Anthropic has "already started" rolling it out, while ChatGPT-maker OpenAI plans implementation "in the coming months," and Microsoft and Google will support the standard beginning Feb 1, 2027.
- While companies like Cloudflare already block bots by default, publishers including the New York Times have pursued legal action over unauthorized scraping; Microsoft noted opting out will not prevent websites from appearing in search results.
Insights by Ground AI
18 Articles
18 Articles
OpenAI, Google, Microsoft and Anthropic commit to a common instruction allowing publishers to block the aspiration of their articles The AI giants
·France
Read Full Article
Open AA, Google, Microsoft and Anthropic have undertaken to provide a unified mechanism for the media and websites.
·Syrian Arab Republic (the)
Read Full Article
The AI giants promised on Friday to the media and other websites a unique way to prevent their content from being sucked into training their models, on the back of a judicial arm with a part of the press. ...
·Brussels, Belgium
Read Full Article
Coverage Details
Total News Sources18
Leaning Left5Leaning Right3Center4Last UpdatedBias Distribution42% Left
Bias Distribution
- 42% of the sources lean Left
42% Left
L 42%
C 33%
R 25%
Factuality
To view factuality data please Upgrade to Premium
