Groq Raises $350M as Nvidia Backs Inference Push

Groq closed a $350 million Series A round Monday, with Nvidia planning to participate alongside lead investor Disruptive. The funding will help customers run medium and large clusters of Nvidia hardware for both training and inference, the company said in a release.

Nvidia signed a $20 billion licensing deal with Groq last year and acquired some of its technology and staff, though Groq continues to operate independently. Alex Davis, Groq's executive chairman and Disruptive's CEO, called inference "the largest and most critical layer of AI infrastructure."

Inference is the stage where a trained AI model takes new data and produces an answer. A customer service chatbot responding to a query or an AI system scanning a financial document both rely on inference. Training a large language model happens once or occasionally. Inference runs every time a user interacts with the system. A single model can handle millions of inference requests each month, each requiring compute power that adds both latency and cost.

Groq says it runs 13 data centers globally, serving more than six million developers, Fortune 500 companies and thousands of AI-native firms. The company raised $650 million in June.

For most enterprises running AI in customer-facing applications, inference performance now matters more than training, according to a separate report last fall. It directly affects user experience, system reliability and operating expenses. The new round gives Groq capital to scale up the clusters that help customers manage that load.