Glossary
Stochastic parrot
Stochastic parrot is a term that Emily Bender, Timnit Gebru and colleagues coined in 2021 for large language models. It means a system that assembles word sequences from its training texts according to probabilities, without these sequences being based on meaning. “Stochastic” means that the next words are chosen according to probabilities, so with a random component. “Parrot” means repeating what has been learned.
An example (made up): A model writes a friendly refusal to a customer. The sentence sounds right because such sentences often appeared in the training texts. Whether the model knows what a refusal means for the customer is the question the term revolves around.
Not to be confused with proof: The term is a warning about harms and responsibility, not evidence of what the model contains. It describes a position in the debate and rests on the argument that bare word sequences, without a connection to the world, cannot learn meaning. Findings showing that more than word sequences can be read from the model (see probe) point the other way. Both sides cite evidence, and the term alone decides nothing.
Where you will meet the term: In the debate about language models, especially in texts about risks and responsibility. The opposing positions are in the lesson Does AI Understand What It Says?.