DCGI flags AI risk: 'Frightening' when apps replace doctors

India's top drug regulator walked through an exhibition at the IIMA Healthcare Summit in Ahmedabad on Saturday and stopped at a stall advertising an “AI Doctor”. Drugs Controller General of India Rajeev Raghuvanshi called the proposition less futuristic and more “frightening”.

“AI is helping in multiple ways. The issues as a regulator today is sometimes too frightening to me – the way I see it moving. Today, whether AI fits into the definition of medical devices is not known to many people,” Raghuvanshi said at the summit themed “Advancing AI in Healthcare”.

The transition is already underway. Raghuvanshi's concern is what happens when AI moves from being a tool used by doctors and drug developers to making recommendations directly to consumers. In conventional medicine, a prescription creates a chain of accountability. A doctor examines the patient, makes a decision and prescribes a drug. If a mobile application tells a consumer what to take, accountability disappears, he argued.

“In drugs there is a prescription system where the doctor writes a prescription and so there is accountability of the doctor. If a mobile application is dictating what to take, where is that accountability and then there is no end to it,” Raghuvanshi added.

The DCGI said regulators approve a product based on defined characteristics of quality, safety and performance and expect those to remain consistent through the product's lifecycle. “The traditional regulator is trained to keep the product the same throughout the lifecycle,” Raghuvanshi said. Regulators are trained to pull a product back to the quality and performance that was approved 10 or 20 years ago. A company wanting to make a significant improvement generally has to return to the regulator with a new application.

AI can change continuously. “It actually changes on a daily basis,” Raghuvanshi said. “I can't pull you back to the original as then the whole concept of AI is lost. So we are also learning and we are also trying to see how to regulate a moving target. It is a global issue that regulators are facing.”

Despite the regulator's fears, AI is already finding a place across the pharmaceutical value chain. Raghuvanshi estimates its adoption remains concentrated among perhaps the top 10% of pharmaceutical companies. For those engaged in discovery research, AI is increasingly becoming an everyday tool.

“The adoption of AI processes in the pharmaceutical industry is actually limited to maybe the top 10 per cent of the companies and those who are involved in discovery-research are using it day-in and day-out because it cuts time significantly. Secondly, a lot of AI is being used in the generic industry to develop algorithms for analysing the data. The real advantage of AI is seen in cutting the timelines and success rates of clinical trials,” he added.

For pharma veteran Pankaj Patel, chairman of Zydus Lifesciences and chairman of the IIMA Board of Governors, AI is beginning to change the economics of an industry that has historically depended heavily on patience, probability and failure. The traditional model involves screening enormous libraries of molecules, most of which fail, over a development cycle that can stretch beyond a decade and cost billions.

AI, Patel argued, is beginning to alter that equation by predicting how molecules could behave, proposing candidates that researchers may not have considered, identifying patients for trials and helping companies recognise failures earlier. “We are not yet at a point where a machine-designed medicine will happen on its own,” Patel said. Research teams equipped with these tools can increasingly accomplish in months what once took years.

According to Patel, an AI scribe can prepare a clinical note while a doctor listens to the patient. A discharge summary that currently consumes hours can be drafted in seconds. A complicated patient history can be summarised before the consultation begins. An AI screening tool can analyse a chest X-ray or retinal scan in a district hospital where a radiologist or specialist may not be available. None of these applications, Patel argued, necessarily looks like a revolution in isolation.

In a healthcare system where patient volumes are rising and clinicians are increasingly burdened by paperwork, the cumulative impact could be significant. “The most valuable thing that AI does is not to diagnose the disease. It is to hand back time to the doctor,” Patel said.

Abbott India's Executive Vice President Vivek Mohan put the same argument in terms of trust. The success of AI in healthcare, he said, will be built around how it is deployed, adopted and trusted, with its primary role being to support clinicians in decision-making rather than replace them. “Some of these AI tools and opportunities may provide preventive approaches, early warning, detections,” he added. Mohan pointed to biosensors that continuously monitor glucose and the billions of data points generated through such devices as a foundation for predictive analysis and more informed conversations between patients and doctors.

The regulatory question remains unresolved. Raghuvanshi's comments suggest the DCGI is still working through how to oversee a technology that changes daily. For an industry racing to embed AI, the risk is that the rules may shift while the tools are already in use.