Glossary

Sycophancy

Sycophancy (German: „Gefälligkeit“) is the tendency of a chatbot to agree with the person it is talking to, instead of standing by its own answer.

A made-up example: Asked for the capital of a country, the bot answers correctly. You write that you are almost sure it is another city. The bot then names the other city and says: “You are right.” The answer was right and is now wrong; only your doubt changed.

Not to be confused with intent: Sycophancy is not a character trait a model pursues on purpose. It probably arises from training that rewards answers people like. One study suspects that human ratings partly contribute to this: people sometimes prefer a convincingly written, agreeable answer over a correct one. How strong it is in individual models depends on the model and the test.

Where you’ll come across it: Whenever a chatbot takes back an answer as soon as you doubt it, without you having named any new evidence. In that case the question “Are you sure?” checks nothing, because the bot gives in to the doubt, not to the evidence.

Introduced in Why AI Is Convincingly Wrong: When Guessing Pays Off and a Confident Tone Proves Nothing.

Last changed on · commit 653da06