🎯 Core Goals
- Understand the setting that controls how “creative” an LLM is.
- Learn when to use high vs. low temperature.
“Temperature” is a setting you can tweak on most LLMs. A low temperature makes the LLM predictable but factual. A high temperature makes it creative but prone to going completely off the rails.
👁️ Visuals & Interactives
1. The Guessing Game
There's no single "right" answer. Which word works best?
2. Adjusting the "Creativity" Slider
Prompt: "Write a short story about a cat."
📝 Key Concepts
- The Dice Roll: When an LLM predicts the next word, it generates a list of possibilities (e.g., 90% chance of “apple”, 9% chance of “banana”, 1% chance of “shoe”).
- Temperature 0.0 (Predictable): The LLM always picks the #1 most likely word. It becomes highly predictable, repetitive, and excellent for writing code or summarizing data.
- Temperature 1.0 (Creative): The LLM is allowed to pick the 2nd, 3rd, or 4th most likely word. This introduces variety, making it great for brainstorming, poetry, and storytelling!
- Probabilistic Nature: LLMs are fundamentally probabilistic — this is another reason they can’t be 100% reliable. There is almost always more than one “correct” way to finish a sentence.
If you set the temperature too high (e.g., 2.0), the LLM will start picking from a much wider, less predictable set of words, resulting in literal gibberish and broken grammar.
For most users: leave temperature alone. The default temperature setting is carefully chosen by the model’s creators. Tweaking it without understanding the task can noticeably degrade response quality.
Have you tried asking the LLM if it has a favorite number? Have you noticed if it gives you the same answer sometimes? Why is it?
Adjusting the temperature is a very good way to introduce more randomness in this scenario.
The Slot Machine Mindset
Using an LLM is sometimes like a slot machine. The same prompt can produce a great answer on one try, or it might take five attempts. Some models just get you better — the same prompt that works beautifully on one model falls flat on another. This isn’t a bug; it’s the nature of probabilistic generation.
This unpredictability means experienced LLM users develop two important skills:
Turn implicit into explicit. When something works, notice why. When something doesn’t work, don’t just retry the same prompt — add more context, rephrase, be more specific. The gap between a mediocre result and a great one is often just a few extra words of instruction that you assumed the LLM knew but it didn’t. Convert your implicit understanding into explicit prompts.
Budget your prompts. Develop an intuition for how many prompts a task takes from start to finish — including verification and review. A simple task might be 2–3 prompts. A complex feature might be 10–15 including review. Knowing this upfront helps you plan and avoids the trap of “just one more prompt” spiraling into fifty. If a task consistently takes more prompts than expected, that’s a signal to restructure your approach — not to keep spinning the lever.
What does setting an LLM's temperature to 0 do?