Time-to-first-token (TTFT) is how long until the model emits its first token, while total latency is how long until the full answer is complete, and the gap grows with output length because generation is roughly linear in tokens produced.
Streaming sends tokens to the client as they are generated, so the user sees output starting at TTFT instead of waiting for the whole response, which dramatically improves perceived responsiveness even though total wall-clock time is unchanged. The cost is complexity: you need Server-Sent Events or WebSockets, chunked handling on the client, and care about not blocking request threads on the server. Interviewers want you to distinguish perceived latency from throughput and to know that for chat-style UX streaming is nearly always worth it, whereas for a batch job that stores a result, streaming buys you nothing.
- Measuring only total latency.
- Buffering the whole response before showing it.