Perplexity accused of scraping websites that explicitly blocked AI scraping

AI firms crawling Nine Entertainment’s news sites 10 times a second

Nine Entertainment thought it had blocked OpenAI, Perplexity and other major AI firms from scraping its content. They kept doing it.

Cloudflare vs. Perplexity: a web scraping war with big implications for AI

When the web was established several decades ago, it was built on a number of principles. Among them was a key, overarching standard dubbed “netiquette”: Do unto others as you’d want done unto you. It’s a principle that lived on through other companies, including Google, whose motto for a period was “Don’t be evil.” The fundamental idea was simple: Act ethically and morally. If someone asked you to stop doing something, you stopped—or at least c…

Perplexity is sneaking onto websites to scrape blocked content, says Cloudflare

Cloudflare has accused the AI company of bypassing websites' no crawl directives. Here's how the CDN is responding.

Perplexity accused of scraping websites even when told not to — here's their response

Perplexity could be in a tricky situation after being called out for accessing websites that don't want to be accessed by AI tools

Perplexity accused of breaking a major online AI scraping rule - but it says it has done nothing wrong

Cloudflare says Perplexity obscures its crawling activity to go undetected when it's presented with a network block.

Cloudflare accuses AI startup Perplexity of evading anti-scraping measures

New Delhi: Cloudflare has accused AI startup Perplexity of bypassing website anti-scraping protections Cloudflare alleged that Perplexity’s web crawlers are systematically ignoring "Robots.txt" directives, a standard protocol used by websites to restrict automated access, and employing sophisticated tactics to scrape content without permission. According to Cloudflare, Perplexity’s crawlers were observed accessing tens of thousands of domains, g…