AIster · The model wars · Products

Claude learns to use a computer

An AI model operates a computer by reading screenshots and sending clicks

Key facts

Date

22 Oct 2024

Era

The model wars

Interest

Products

Threads

Agents

Who

Anthropic

What

Computer use, public beta, with an upgraded Claude 3.5 Sonnet

OSWorld

14.9% screenshot-only (22.0% with more steps); next-best system 7.8%

Anthropic released computer use in public beta (an early version open to developers): Claude 3.5 Sonnet could look at a screen, move a cursor, click buttons and type text. Anthropic called it experimental and at times cumbersome and error-prone, but it was an early step towards agents (AI systems that carry out tasks on their own) that operate ordinary software.

Sources

1. Anthropic, Introducing computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku, 2024-10-22
2. Anthropic (Claude Docs), Computer use tool, 2026-10-01