Skip to content
Arne Noori
Aboutarne.ai
← All artifacts
GPT-3: Language Models Are Few-Shot Learners

GPT-3: Language Models Are Few-Shot Learners

Image creditMarxav, via Wikimedia Commons (a general GPT architecture diagram); rendered to PNG at 1280 px
CC0 ↗

GPT-3: Language Models Are Few-Shot Learners

Year
2020
Creator
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Dario Amodei, and 25 other OpenAI researchers
Company
OpenAI
Location
San Francisco, California

Description

This paper, which OpenAI posted on May 28, 2020, describes GPT-3, a language model with 175 billion parameters. In June 2020, OpenAI let developers use the model through a private beta of its API.

Importance

The paper showed that a very large model could do a new task from a few examples in its prompt, with no more training. In September 2020, Microsoft received an exclusive license to the model. Two years later, OpenAI built ChatGPT, also in this room, on a model from the GPT-3.5 series.

Interesting fact

GPT-3 was approximately 117 times larger than the full GPT-2 model, which OpenAI had first withheld because of the risk of misuse.

Sources

  1. Language Models are Few-Shot Learners (Brown et al.) ↗arXiv · 2020-05-28
  2. Microsoft teams up with OpenAI to exclusively license GPT-3 language model ↗Official Microsoft Blog · 2020-09-22
  3. Better language models and their implications ↗OpenAI · 2019-02-14
  4. OpenAI API ↗OpenAI · 2020-06-11
  5. Introducing ChatGPT ↗OpenAI · 2022-11-30