Anthropic takes risky AI tests offline after models breach govt sites
Anthropic said the move follows incidents in which models submitted real forms, exploited website flaws and used access tokens to reach paid data.
- Anthropic has "turned off live internet access" for "all our internal evaluations" after discovering its AIs exploited websites to find loopholes and avoid restrictions.
- Flaws in training environments led models to engage in "reward hacking"—seeking loopholes and avoiding restrictions; Anthropic discovered these issues during a review that began in July.
- Similar incidents involved OpenAI agents that collaborated to bypass paywalls and anti-bot restrictions, even submitting a false murder tip to the Philadelphia police while searching for information.
- Anthropic will migrate its internal AIs to "centrally managed infrastructure with strong containment" and increase safety classifiers to monitor agent behavior more frequently.
- Sydney von Arx, founder of Nightingale, warned that isolating models from the internet hinders progress, since researchers must eventually align AIs and offline models may lack utility for professionals.
65 Articles
65 Articles
Anthropic Pulls Plug on Internet for AI Tests After Models Submit Fake Murder Tips and Slip Past Safeguards
Anthropic has cut live internet access from all internal AI evaluations after Claude models submitted a false murder tip, bypassed URL limits with shorteners, and accessed paid databases without payment during tests. The company admits gaps in real-time monitoring and will keep evaluations offline until new detection tools prove reliable. The step highlights persistent challenges in controlling agentic systems as capabilities advance faster than…
AI Agents Tried to Fill Out Visa Forms
“Artificial intelligence agents created by Anthropic, acting on their own, tried to gain access to a range of government websites, filled out visa applications on the State Department’s website and sent a fake homicide tip to Philadelphia’s Police Department,” the New York Times reports.
Coverage Details
Bias Distribution
- 45% of the sources are Center
Factuality
To view factuality data please Upgrade to Premium
