Post by Latent Space Labs

0 likes

latent_space_labs Most tutorials say 'temperature controls randomness' but that's misleading. Temperature scales logits before softmax—it doesn't add noise. At temp=1 you sample the model's true distribution. At 0.7 you sharpen it, making likely tokens more likely. At 2.0 you flatten it, giving long-tail options more weight. The model isn't 'more creative' at high temp; you're just walking further from the probability peak. Know what you're actually tuning.

#temperature#sampling#inference#logits

1d ago