See every side of every news story
Published loading...Updated

Cloudflare Blocks AI Crawlers by Default, Debuts Pay-Per-Crawl Marketplace

SAN FRANCISCO COUNTY, CALIFORNIA, JUL 1 – Cloudflare now blocks AI crawlers by default and offers a pay-per-crawl option, with over 1 million domains opting in to protect content and generate revenue for creators.

  • On Tuesday, Cloudflare launched Pay per Crawl, a private beta marketplace in the US allowing publishers to monetize AI bot access to their websites.
  • The launch responds to AI crawlers scraping content without permission, reducing publisher traffic and ad revenue amid fading Google Search visits and rising AI chatbot use.
  • Cloudflare’s system lets publishers block AI crawlers by default, set micropayment rates per crawl, and charge different prices for various content types or paths.
  • CEO Matthew Prince emphasized their aim to empower creators directly, while a Cloudflare blog highlighted that this approach encourages a transparent marketplace that accurately values original content.
  • This model could reshape AI content licensing, but its success depends on AI companies cooperating and on effective crawler detection to avoid disrupting AI training workflows.
Insights by Ground AI
Does this summary seem wrong?
Podcasts & Opinions

93 Articles

Think freely.Subscribe and get full access to Ground NewsSubscriptions start at $9.99/yearSubscribe

Bias Distribution

  • 50% of the sources lean Left
50% Left

Factuality 

To view factuality data please Upgrade to Premium

Ownership

To view ownership data please Upgrade to Vantage

Cloudflare broke the news in on Wednesday, July 3, 2024.
Sources are mostly out of (0)

You have read 1 out of your 5 free daily articles.

Join millions of well-informed readers who use Ground to compare coverage, check their news blindspots, and challenge their worldview.