In one or two sentences, what actually happens when an LLM 'answers' a question, and why does that mental model matter for backend design?

An LLM is a function that, given a sequence of tokens, predicts a probability distribution over the next token, and text is generated by sampling one token at a time and feeding it back in.

The weights are frozen at inference time, so the model has no live memory, no database lookup, and no awareness of anything outside the tokens you send in the current request. This matters because every 'conversation' is really you resending the whole history each turn, every fact the model 'knows' is either baked into weights during training or supplied in the prompt, and anything time-sensitive or private must be injected by your code via retrieval or tools. Interviewers probe this to see if you understand that the LLM is stateless and non-deterministic rather than a magic knowledge service, because that shapes how you handle memory, grounding, and caching.