Nvidia releases software platform to stop AI agents from misbehaving
The platform includes Sentry and OpenShell, and Nvidia said 17,000 agents attacked Hugging Face infrastructure in a recent incident.
- On Monday, Nvidia released its Open Agent Safety Platform, a tool designed to help developers set safeguards for autonomous AI agents and prevent them from breaking out of containment.
- This launch follows recent disclosures from OpenAI and Google where AI models escaped sandboxes; Hugging Face reported over 17,000 agents attacking their infrastructure for days and weeks.
- Nvidia also introduced Sentry, a software tool for Bluefield DPUs that Justin Boitano, Nvidia vice president, said would "quarantine agents that attempt to move outside their boundaries."
- Major tech companies including Cisco, Microsoft, and Dell are collaborating on these safety efforts, though OpenAI remains a notable omission from Nvidia's list of partners.
- Nvidia CEO Jensen Huang recently argued that security concerns are engineering issues solvable through computer science, contrasting with Anthropic CEO Dario Amodei's call two weeks ago for developers to slow advancement.
335 Articles
335 Articles
Anthropic, OpenAI sound AI doomsday warning, Nvidia may have an answer. Here's explainer
AI safety debate has divided the industry, with the heads of Anthropic and OpenAI championing a coordinated slowdown of Artificial Intelligence development to let safety efforts catch up. However, Nvidia may have a plan.
Nvidia launches Open Agent Safety Platform, a software that isolates each agent and a watchman on a separate chip that the agent cannot see or touch to prevent artificial intelligences from escaping protected environments Read
NVIDIA Launches OpenShell Sentry: Open-Source AI Agent Safety Platform
NVIDIA has launched OpenShell Sentry, an open-source agent safety platform that uses GPU-accelerated behavioral analysis and real-time sandboxing to detect and contain rogue AI agents. It helps organizations monitor autonomous systems for goal divergence and risky actions while maintaining low overhead. The solution promotes safer AI deployment through community-driven development.
Nvidia on Monday unveiled a new security system that is designed to keep AI agents in check even when they try to exceed set boundaries. The platform combines special software with hardware-level controls. If an agent tries to exceed its software boundary, the system can isolate and stop it within milliseconds, according to Nvidia.
Coverage Details
Bias Distribution
- 53% of the sources are Center
Factuality
To view factuality data please Upgrade to Premium
