AIster · The ChatGPT shock · Launches

OpenAI releases GPT-4

The model everyone else measured themselves against for a year

Key facts

Date

14 Mar 2023

Era

The ChatGPT shock

Interest

Launches

Threads

Model wars

Who

OpenAI

What

GPT-4, a multimodal model taking image and text input

Benchmark

Simulated bar exam about 90th percentile (GPT-3.5 about 10th)

Same day

Anthropic introduced Claude and Claude Instant

On 14 March 2023 OpenAI announced GPT-4, calling it “the latest milestone in OpenAI’s effort in scaling up deep learning.” It was a large multimodal model: it accepted image and text inputs and produced text. OpenAI said it had spent six months aligning GPT-4, meaning training it to behave as intended, using lessons from adversarial testing (deliberate attempts to make it misbehave) and from ChatGPT.

The exam results

The headline claim was about professional and academic exams. OpenAI reported that GPT-4 passed a simulated Uniform Bar Exam with a score of 298 out of 400, around the 90th percentile of test takers, while GPT-3.5 scored 213, around the 10th. On the LSAT, GPT-4 was estimated at around the 88th percentile against GPT-3.5's 40th. OpenAI noted that some exam questions had been seen in training but said it believed the results were representative.

What OpenAI did not say

The accompanying technical report withheld the details researchers usually expect. Citing “the competitive landscape and the safety implications of large-scale models like GPT-4”, it said it contained no further details about the architecture (including model size), hardware, training compute, dataset construction or training method.

How people got it

GPT-4's text input went to ChatGPT Plus subscribers with a usage cap, and to developers through an API waitlist. Image input was held back and tested with a single partner first. OpenAI also released Evals, an open-source framework for testing model performance.

The same day

Anthropic introduced Claude the same day, offered in two versions, Claude and Claude Instant, after a closed alpha with partners including Notion, Quora and DuckDuckGo. The two launches on one day showed how crowded the field was becoming.

Why it matters

GPT-4 became the reference point for the models that followed in this era, and its exam results were widely cited as evidence that language models could handle skilled professional tasks.

Questions people ask

When was GPT-4 released?

OpenAI announced GPT-4 on 14 March 2023, making it available to ChatGPT Plus subscribers and, via a waitlist, to developers on the API.

Did GPT-4 really pass the bar exam?

OpenAI reported that GPT-4 scored 298 out of 400 on a simulated Uniform Bar Exam, around the 90th percentile of test takers. It was a simulated exam run by OpenAI, not an official sitting.

Sources

1. OpenAI, GPT-4, 2023-03-14
2. arXiv, GPT-4 Technical Report, 2023-03-15
3. Anthropic, Introducing Claude, 2023-03-14