Anthropic Teaches Its Claude LLM to Use Computers Like We Do — with a Virtual Keyboard and Mouse

Artificial intelligence specialist Anthropic has announced new entries of its Claude large language model (LLM) family — and a feature that allows the latest Claude 3.5 Sonnet to take control of your computer, interacting with other software using a virtual keyboard and mouse.

"Available today on the API [Application Programming Interface], developers can direct Claude to use computers the way people do — by looking at a screen, moving a cursor, clicking buttons, and typing text," Anthropic explains of its model's new capabilities. "With computer use, we're trying something fundamentally new. Instead of making specific tools to help Claude complete individual tasks, we're teaching it general computer skills — allowing it to use a wide range of standard tools and software programs designed for people."

The new feature, available in beta with the warning that it's still very much an experimental offering that is "at times cumbersome and error-prone," is exclusive to the company's upgraded Claude 3.5 Sonnet model. This, Anthropic claims, delivers a jump in benchmark results for reasoning, code generation, math problem solving, and visual question-and-answer responses — outperforming not only the previous Claude 3.5 Sonnet model but those from competitors too.

At the same time, Anthropic unveiled the new Claude 3.5 Haiku, designed as a successor to Claude 3 Haiku. "For the same cost and similar speed to Claude 3 Haiku," the company claims, "Claude 3.5 Haiku improves across every skill set and surpasses even Claude 3 Opus, the largest model in our previous generation, on many intelligence benchmarks."

The upgraded Claude 3.5 Sonnet is available now, complete with the computer-use beta; Claude 3.5 Haiku is due to launch later this month as text-only model, with image input to follow in the near future. More information is available on the Anthropic website.