Ask HN: Why is Generative AI non-deterministic

by michaeljx on 4/1/2025, 9:06:01 PM with 7 comments
Given a trained LLM model with fixed weights, why is it that the same prompt yields different responses? Or is it the case that some type of RL takes place?

  • by bigyabai on 4/1/2025, 9:10:36 PM

    Generative AI typically is deterministic, most inference software includes a random seed to yield different results on each repeated entry.

  • by schoen on 4/1/2025, 9:10:20 PM

    It's deliberately made nondeterministic, partly using something called softmax

    https://en.wikipedia.org/wiki/Softmax_function

    I'd say mainly in order to avoid boring its users.