This AI shortcut threatens to destroy the industry's profits

AI distillation has moved from a research curiosity to a direct threat to the income statements of the industry's biggest model builders. OpenAI and Google each spend billions on training their largest models. Anthropic, a smaller rival, also invests heavily. Smaller companies now replicate those models for a fraction of the cost, using a technique that extracts the knowledge from a larger model into a smaller one.

The process is straightforward. A company runs a large model on a massive dataset and records its outputs. It then uses those outputs to train a smaller model. The smaller model can match the larger one on many tasks. The cost difference is huge. Training GPT-4 cost an estimated $100 million. A distilled version of a similar model can be built for under $1 million, researchers said.

The profit model of the leading AI labs depends on charging for access to their models, either through API calls or subscriptions. If any competitor can offer a near-equivalent model for a tenth of the price, the pricing power collapses. This is not a theoretical risk. Distilled models already exist for several open-source and closed-source models. Meta's Llama has been distilled multiple times. The same is happening with OpenAI's GPT series, though the company tries to block it.

The next catalyst is a court case or a licensing change. OpenAI's terms of service prohibit using its outputs to train competing models. Enforcement is difficult. A public legal fight could set a precedent. Some customers are already evaluating distilled models.

The readthrough extends beyond the model builders. NVIDIA's revenue from training chips may face headwinds if distillation reduces the need for massive training runs. Inference demand is rising. Smaller models are proliferating. The net effect is unclear.

This dynamic echoes what Mark Cuban recently argued about AI labs and platforms. He said the real value is in the application layer, not the model itself. That argument looks stronger with distillation in play. Why Cuban says AI labs can't replace Lovable, Replit yet

NVIDIA's dominance in training hardware is a key variable. If distillation compresses the training market, NVIDIA's growth rate would slow. NVIDIA profile

Microsoft and Google report second-quarter earnings in July. Both will face questions about competitive pressure from distilled models.