Published 9 hours ago • loading... • Updated 44 minutes ago

Cloudflare Blocks AI Crawlers by Default, Debuts Pay-Per-Crawl Marketplace

On Tuesday, Cloudflare launched Pay per Crawl, a private beta marketplace in the US allowing publishers to monetize AI bot access to their websites.
The launch responds to AI crawlers scraping content without permission, reducing publisher traffic and ad revenue amid fading Google Search visits and rising AI chatbot use.
Cloudflare’s system lets publishers block AI crawlers by default, set micropayment rates per crawl, and charge different prices for various content types or paths.
CEO Matthew Prince emphasized their aim to empower creators directly, while a Cloudflare blog highlighted that this approach encourages a transparent marketplace that accurately values original content.
This model could reshape AI content licensing, but its success depends on AI companies cooperating and on effective crawler detection to avoid disrupting AI training workflows.

Insights by Ground AI

Does this summary seem wrong?

33 Articles

All

Left

Center

Right

PC Mag

Lean Left

Cloudflare to Block AI Crawlers From Scraping Websites Unless They Pay

As publishers grapple with declining traffic thanks to AI, Cloudflare changes the default to 'block AI crawlers unless they pay creators' via a new system it's calling 'pay per crawl.'

3 hours ago·United States

Read Full Article

Observador

Lean Right

Cloudflare Launch Functionality to Prevent Web Content From Being Collected by Ia Boots

Cloudflare, online traffic management technology, launchs a feature that will allow you to automatically block the boots that collect data from IA training sites without permission.

7 hours ago·Portugal

Read Full Article

Engadget

Lean Left

Cloudflare experiment will block AI bot scrapers unless they pay a fee

Every new domain customer that signs up with Cloudflare to manage their website traffic will now be asked if they want to allow AI crawlers or to block them altogether.

7 hours ago·United States

Read Full Article

Market Screener

Center

Cloudflare Rolls Out Tool for Website Owners to Block AI Crawlers, Charge for Content Scraping

Cloudflare said Tuesday it has rolled out a new tool using which website owners can prevent AI crawlers from accessing their content without permission or compensation, by default. AI companies...

7 hours ago

Read Full Article

Press Gazette

Center

Major UK and US publishers join forces to block AI scrapers

Internet infrastructure provider Cloudflare is now blocking all AI scrapers accessing content by default in an industry first. The move has been backed by more than a dozen major news and media publishers including the Associated Press, The Atlantic, Buzzfeed, Conde Nast, DMGT, Dotdash Meredith, Fortune, Gannett, The Independent, Sky News, Time and Ziff Davis. Cloudflare said its default setting for new domains is now to block AI crawlers that d…

8 hours ago

Read Full Article