Gradium CEO: Voice AI Needs Context, Not Just Words

Voice AI that transcribes every word accurately can still fail completely. The transcript is right. The experience is wrong. Anyone who has argued with a voice assistant knows the gap.

That gap becomes a business problem when voice moves from dictation to instructing AI agents. The platform has to know who is speaking, what came before, what the person means in that moment and whether the words are a conversation or a command.

Neil Zeghidour, CEO of Paris-based voice AI startup Gradium, has been building toward that problem. When he spoke with PYMNTS CEO Karen Webster last year, the company had just raised a $70 million seed round. This year Gradium added another $30 million, with Nvidia among the investors. Total funding is $100 million.

“What I hadn’t anticipated is how much of the interaction with computers would become more and more voice-driven,” Zeghidour said during the PYMNTS “Monday Conversation” with Webster. Software development and AI research, he added, are increasingly becoming “more conversations than a set of instructions.”

The shift changes the standard voice AI has to meet. Speed, cost and a natural-sounding voice still matter. The real test is whether the system can understand the situation and the sentiment of the person on the other end, connect to the right information and safely turn a conversation into action.

Voice technology has long been judged as a transcription tool. Can it recognize the words? Can it handle an accent? Can it respond without an awkward delay? Those benchmarks made sense when the job was turning speech into text. They are incomplete when the listener is an agent.

An agent has to understand meaning, not simply preserve the words. The same word can refer to a product, a person, a place, a request or an instruction. Humans sort through that ambiguity by using the rest of the conversation, what they know about the speaker, what is happening around them, the way the words were spoken and the task at hand. A voice agent needs the digital equivalent of those cues to know the difference between “bye” and “buy.”

“All the failure cases we see are a lack of context,” Zeghidour said. “The whole thing is how we can communicate this context to the agent.”

A system scheduling a medical appointment may misinterpret an unfamiliar surname unless it can check the hospital’s physician directory. An agent handling an order needs the customer’s account history, the product catalog, current inventory and the company’s policies. It also needs to know whether the person speaking is allowed to approve a refund, move money or change an account.

That is why a beautiful voice can still produce a bad experience. As Webster put it, “I don’t really care if it’s an AI [agent] speaking back to me if I’m getting good information.” A chatbot that sounds human but gives a generic answer, forgets what was said two turns earlier or sends the user to a person anyway has not solved the problem.

Part of the problem, Webster and Zeghidour agreed, is that voice looks easier to build than it is. The basic pieces are widely available: speech recognition, a language model and speech synthesis. Companies see those components and assume they can connect them on their own.

That can produce an impressive demo in a quiet room with one speaker and a predictable request. Production is different. People interrupt, change direction, use company-specific terms, refer to something mentioned minutes earlier and expect the system to know when they have finished talking. The agent has to respond quickly, use the right data, preserve the thread of the conversation and recover gracefully when it is confused.

A failure anywhere in that chain is experienced as one failed conversation. The customer does not separate the speech model from the language model or the connection to the company’s software. They experience a voice assistant that is slow, repetitive, unhelpful or wrong. That is why so many voice experiences create frustration even when the transcription itself is accurate.

Zeghidour puts reliability at the base of the voice AI hierarchy. “If you look at the Maslow pyramid of people making agents, the very foundation of it is: It needs to be reliable,” he said. “As long as you have reliability, then you can start thinking about naturalness, and is the voice nice, and is the flow natural.”

That order matters. Companies may be tempted to focus on whether the voice sounds warm or human. Naturalness can make a weak system more disappointing because it raises expectations the underlying experience cannot meet. The harder work is making sure the assistant understands, remembers and completes the task.

Zeghidour said the challenge becomes much harder as voice assistants move off the desktop and into portable devices. A call-center system at least knows that the person on the line is speaking to it and that the environment is usually quiet. A portable assistant may be listening in a car, on a sidewalk, in a restaurant, at an airport or in a room where several people, a television and other devices are all making noise. Real life is noisy.

A voice AI system has to isolate the person it serves, distinguish an instruction from an overheard conversation and decide when a request begins and ends. It may also need to follow the same person as the surroundings change. Humans do this selective listening automatically, and even humans have trouble following one voice in a crowded room.

Noise cancellation alone does not solve that problem. A device can transcribe every voice in the room accurately and still fail its user if it cannot identify which words are directed at the agent and which should be ignored. Context includes not only the conversation and the enterprise data behind it, but also the speaker, the device, the location and the moment.

“Fundamentally, the way we are going to interact with machines is a conversation through natural language,” Zeghidour said. “Instead of us adapting to the language of the machine, the machine is now speaking our language.” For that promise to hold outside a controlled setting, the machine also has to know when our language is meant for it.

The stakes rise again when voice moves from answering a question to completing a transaction. Webster recalled early dictation software that sometimes treated an ordinary word such as “send” written in the body of an email as a command to send the email and dispatched an unfinished email. Modern transcription is far better, but an agent creates a more consequential version of the same risk. It has to distinguish between talking about a purchase and telling software to make one.

As Webster noted, the issue becomes more serious when “voice becomes the agent command to complete a transaction.” A voice system connected to money, medical information, workplace software or a customer account has to know not only what was said, but also whether the speaker intended an action and the agent had the authority to approve it.

Zeghidour said enterprises will need permission structures around voice actions. A low-risk request may happen immediately. A higher-risk action may require the agent to repeat the instruction, confirm the details, verify identity or hand the decision to a person. Context can also tell the system when a confused or frustrated customer should be moved to a human before the experience gets worse.

This is the larger significance of Gradium’s $100 million bet. Voice is not simply becoming a more natural way to operate software. Zeghidour believes voice will become one of the most important developments in the AI economy. Not because AI can talk like a person, but because people will be able to talk to an agent as naturally and effectively as they talk to another human.

Nvidia’s investment in Gradium signals that the hardware giant sees voice AI as a growth vector for its AI chips. NVDA stock page shares rose 2.93% to $200.75 on the session, with an Alpha Score of 71 out of 100.

Zeghidour and Webster both noted that the companies solving the context problem will have built much more than a better transcript. They will have built the interface for the next generation of enterprise software.