Stocks● Neutral
OpenAI Pushes ChatGPT Into the Browser as an AI Agent
OpenAI launches Computer Use tools for ChatGPT, letting the AI take over a browser to automate tasks. Rivals are racing to catch up, but security and speed remain challenges.
Continue with
OpenAI is pushing its long-standing ambition to build an AI agent that can use a web browser into its flagship product. The company has launched a Chrome extension, created cloud and in-app browsers for ChatGPT, and added Computer Use tools to its Codex coding environment. The underlying technology lets ChatGPT take over a user's browser to complete tasks.
Ari Weinstein, a manager on OpenAI's Computer Use team, told Business Insider the goal is to change how people interact with their computers. "Once ChatGPT can use computers and software faster than you or I can, it's going to change the way that you, by default, want to interact with your computer," he said.
Rival Anthropic was first to market with its own Computer Use feature in 2024. As of May, at least 600,000 organizations had tried Anthropic's Claude Cowork feature. OpenAI now says its tech has caught up, and it is betting that users will hand over more work to ChatGPT.
The training process for OpenAI's newest models included steps that improved their computer-use abilities. Zhou Yu, an AI researcher at Columbia University who worked on a benchmark test for computer-use agents, told Business Insider that the company likely included frame-by-frame screenshots in the earliest phase of model training. In a later step, developers provide the models with data showing the ideal inputs for a given output, such as filling out a tax form or 3D-modeling an object.
OpenAI has also upgraded how its models interact with a computer. Previously, ChatGPT would take a screenshot, analyze pixels, inject a command, and repeat. Now, when the tool has a website open, it can read the page's memory structure, accessibility information, and link connections directly. Weinstein said the mix of tools provides Codex with a way to test code, find bugs, and make improvements in a "full loop."
Cristian Medina Ruiz, a hobbyist coder in the Czech Republic, used Codex to rebuild the 2013 city-building game "SimCity" from its original source code. The Computer Use tool opened a window to verify that the game correctly simulated car movements and building images. Ruiz said he now leaves his code running and iterating when he is away from his computer.