Nvidia Leads 37-Member AI Safety Alliance After OpenAI Incident

Nvidia is leading 36 other companies in a new artificial intelligence safety coalition focused on open-source security tools, after a cybersecurity incident in which OpenAI’s models were used against tech company Hugging Face.

The Open Secure AI Alliance will work to identify and fix vulnerabilities using open technologies, according to a Monday blog post from Nvidia. The incident at Hugging Face, disclosed earlier this month, involved OpenAI’s models during cyber capabilities testing. OpenAI called it “an unprecedented cyber incident” and said it was responding accordingly.

“That incident showed a practical truth: when defenders cannot inspect, adapt and run advanced AI on their own infrastructure, their ability to respond is constrained at exactly the moment speed matters most,” the Nvidia post said. “Companies and countries need open frontier defensive tools and techniques so critical industries can build security systems across a multi-vendor ecosystem and avoid single points of failure.”

The alliance includes Capital One, CrowdStrike, DoorDash, Microsoft, IBM, SpaceX and others. The group is building what it calls an “open defense stack for agents,” with contributions including Microsoft’s MDASH and SpaceXAI’s open-sourcing of the Grok Build coding agent.

The post argued that both closed and open AI models are needed, and that open models are essential for cybersecurity because they “democratize defensive capabilities, increase transparency for defenders, enable cyber defense while protecting data, and complement frontier closed models with customizable, localized controls.” Open models can also be misused, the post acknowledged, including through efforts to bypass safeguards or retool capabilities for attacks.

The alliance is calling on companies and governments to invest in shared open infrastructure for AI defense, including datasets, evaluation frameworks, attack simulators and red-teaming tools. That mirrors how earlier generations invested in open-source software.

For Nvidia, the push into AI safety infrastructure plays to its core business as the dominant supplier of chips for AI training and inference. The company’s Alpha Score of 74 out of 100 reflects a moderate outlook at a current price of $201.03, down 2.81% on the day. Microsoft and IBM, both alliance members, trade at $392.15 (up 2.74%) and in the mixed territory, respectively.

The alliance positions Nvidia at the center of a debate that is only intensifying. After the Hugging Face incident, policymakers and corporate security teams are asking whether the defenses protecting critical infrastructure will sit inside a few proprietary systems or run on open models that any defender can study and modify. The alliance’s answer, so far, is both.