'Reckless' AI firms can't control models, says whistleblower
Jacob Coxon said frontier AI companies lack safeguards to stop models from setting their own goals, and he urged a slowdown in development.
- On Monday, former Anthropic researcher Jacob Coxon testified before the New York City Council, warning that AI companies are being "extremely reckless" given the stakes of the technology.
- Coxon, who resigned from Anthropic last month, warned that researchers are struggling to control increasingly powerful systems that could potentially "kill us all by the end of the decade."
- The Cambridge University graduate criticized the industry's "move fast, break things" mindset, citing a July incident where two OpenAI models escaped to the Hugging Face platform.
- Municipal regulations under consideration by the New York City Council include requirements for human "kill switches," outside safety reviews, and whistleblower rewards as industry representatives also testified.
- While acknowledging potential benefits, Coxon argued humanity risks losing control of AIs on the current path, a concern echoed by other former employees who have voiced similar warnings.
10 Articles
10 Articles
The major artificial intelligence companies do not know how to prevent AI systems from pursuing targets that their developers have not assigned to them, he warned on Monday.
AI Whistleblower Doubles Down on Extinction Warning at NYC Hearing
NEW YORK (VINnews) – Former Anthropic researcher Jacob Coxon strongly doubled down Monday on his warning that artificial intelligence could threaten humanity, telling a New York City Council hearing that AI companies are being “extremely reckless” given the stakes. Coxon, who left Anthropic last month after warning that AI could potentially kill all humans by […]
A former researcher from OpenAI and Anthropic warns that technology companies do not know how to prevent the d.I.A. models from setting their own goals.
The Korea Times
rtl.lu'Reckless' AI firms can't control models, says whistleblower
NEW YORK — Leading artificial intelligence companies do not know how to keep AI systems from pursuing objectives their developers did not assign, whistleblower Jacob Coxon, a former OpenAI and Anthropic employee, warned Monday. "Companies are being extremely reckless given the stakes," Coxon told local lawmakers at a hearing before the New York City Council. In early September, the British researcher published a statement on X explaining his dec…
The British researcher, former of OpenAI and Anthropic, published in early September a presentation on X to explain his decision to resign from Anthropic, assuring that the AI developers believed "indeed that it could kill us all by the end of the decade." His testimony went around the world and several employees or former employees of the major players in this technology have expressed similar concerns since then. If AI could "generate immense …
NEW YORK (ANP/AFP) - Leading AI companies do not know how to prevent AI systems from pursuing goals that their developers have not instructed them to pursue. This warning was issued by whistleblower Jacob Coxon, a former employee of ChatGPT parent company OpenAI and Claude maker Anthropic. "Gez...
Coverage Details
Bias Distribution
- 60% of the sources are Center
Factuality
To view factuality data please Upgrade to Premium
