AIster · The ChatGPT shock · Launches

Meta's LLaMA leaks and soon runs on a MacBook

The leak and llama.cpp helped start the open-weights movement

Key facts

Date

24 Feb–10 Mar 2023

Era

The ChatGPT shock

Interest

Launches

Threads

Open vs closed

Who

Meta AI; Georgi Gerganov (llama.cpp)

Models

LLaMA 7B, 13B, 33B and 65B, noncommercial research licence

Leak

Torrent posted on 4chan on 3 Mar 2023

llama.cpp

Published 10 Mar 2023; 4-bit LLaMA-7B on a MacBook

On 24 February 2023 Meta announced LLaMA, a family of large language models in four sizes: 7B, 13B, 33B and 65B parameters (the values a model learns in training; B stands for billion). Unlike ChatGPT, it was not a chatbot. Meta released the models under a noncommercial licence for research use, with access granted case by case to academics, government and civil-society groups and industry labs. Its reasoning was that smaller, well-performing models would let researchers without large computing budgets study how these systems work.

The leak

Access did not stay controlled for long. According to The Verge, a downloadable torrent of the model was posted on 4chan on 3 March, about a week after Meta began taking requests, and spread across AI communities. One researcher who compared the leaked files with the official release said they matched. Meta would not discuss the origin of the leak, but said in a statement that some people had tried to circumvent the approval process.

Reaction split along a familiar line. Some warned the leak would lead to spam and phishing at scale. Others argued that open access was needed to find and fix weaknesses, and that it would stop the technology from being held only by a few companies.

Running on a laptop

The leak mattered because of what people did next. On 10 March developer Georgi Gerganov published llama.cpp, a plain C/C++ program whose stated goal was “to run the model using 4-bit quantization on a MacBook.” Quantisation shrinks a model by storing its numbers with less precision. Its README admitted it “was hacked in an evening” and showed the 7B model writing a list of steps for building a website on Apple silicon, with no GPU server involved.

Until then, running a capable language model meant renting data-centre hardware or using someone else's API (an online service that runs the model for you). Within a week of the leak, a capable model could run on a laptop.

Why it matters

The LLaMA leak and llama.cpp helped start an ecosystem of open-weights models (ones whose files anyone can download), fine-tuned versions and local AI tools. It also pushed Meta towards a deliberate open strategy: its next model, Llama 2, was released for commercial use in July 2023.

Questions people ask

Did Meta's LLaMA model leak?

Yes. Meta released LLaMA to approved researchers on 24 February 2023, and on 3 March a torrent of the weights was posted on 4chan and spread widely.

What is llama.cpp?

An open-source C/C++ program first published in March 2023 that runs LLaMA-family models on ordinary computers, starting with 4-bit quantised (compressed) models on Apple silicon MacBooks.

Sources

1. Meta AI, Introducing LLaMA: A foundational, 65-billion-parameter large language model, 2023-02-24
2. The Verge, Meta’s powerful AI language model has leaked online — what happens now?, 2023-03-08
3. GitHub, llama.cpp README (11 Mar 2023 revision), 2023-03-11