When using AI APIs, we’re told to "increase the temperature for more creative responses" and "decrease it for more deterministic answers."
It sounds like a complex behavioral prompt.
It’s actually just a single division operation.
The Raw Output: Logits
Before an LLM outputs a word, it outputs logits.
Logits are raw, unnormalized numbers representing the model's confidence for every possible next token in its vocabulary. They can be positive, negative, large, or small.
# Raw model output (logits) for 3 possible next words: logits = [2.0, 1.0, -1.0]
To turn these raw numbers into probabilities (percentages that sum to 1.0), the model passes them through a softmax function.
probabilities = softmax(logits) # Results: [0.70, 0.26, 0.04]
The model then randomly samples from this distribution. It will pick the first word 70% of the time, the second word 26% of the time, and the third word 4% of the time.
Where Temperature Comes In
Temperature is applied right before the softmax.
You simply divide the logits by the temperature value.
logits = logits / temperature probabilities = softmax(logits)
That’s it. That is the entire mechanism. But the math of how division affects the softmax function is what creates the "creativity" effect.
Scenario 1: Temperature = 1.0 (Default)
Dividing by 1 changes nothing. The model uses its raw confidence.
logits = [2.0, 1.0, -1.0] probs = [0.70, 0.26, 0.04]
Scenario 2: Temperature = 0.5 (Deterministic / Safe)
Dividing by a number less than 1 makes the logits larger. Softmax exaggerates large differences. The most likely options become overwhelmingly probable, and the unlikely options get crushed to zero.
logits = [2.0, 1.0, -1.0] / 0.5 = [4.0, 2.0, -2.0] probs = [0.88, 0.12, 0.00]
The model is now playing it incredibly safe. It will almost always pick the top choice.
Scenario 3: Temperature = 2.0 (Creative / Chaotic)
Dividing by a number greater than 1 makes the logits smaller, pushing them closer together. Softmax treats similar numbers as similar probabilities. The distribution flattens out.
logits = [2.0, 1.0, -1.0] / 2.0 = [1.0, 0.5, -0.5] probs = [0.47, 0.28, 0.25]
Now, the model is much more likely to pick the second or third choice. It takes risks. It hallucinates. It gets "creative."
The Takeaway
Temperature does not make the model "think" differently. It does not change the model's internal logic.
It just flattens or sharpens the final probability distribution.
T < 1: Sharpens the distribution. The rich get richer. (Use for code generation, facts, JSON extraction).
T > 1: Flattens the distribution. Evens the playing field. (Use for brainstorming, creative writing, varying outputs).