LLMOps interview questions

LLMOps interviews test whether you have run something rather than built something. The questions cluster around the parts that only exist once real traffic arrives: reproducibility, regression detection, cost, and what you did when it broke.

Most candidates can describe an architecture. Far fewer can describe an incident, which is why the incident question carries disproportionate weight.

The underlying concern is whether you will be surprised by things an experienced operator expects. Someone who already knows that providers change behaviour silently, that cost is fine in staging and alarming in production, and that quality degrades without any error being thrown needs far less supervision.

Expect a question about versioning. The trap is answering as if this were classical ML, where you control the artefact. With a hosted model you do not: the provider can change behaviour with no deployment on your side.

A strong answer captures everything around the call — prompt, context, parameters, retrieval results, model identifier — precisely because the one component you cannot snapshot is the model. Mentioning that you pin model versions where the provider allows it, and monitor for drift where it does not, shows you have thought about the gap rather than ignored it.

The related question is what you would do the week a pinned version is deprecated. Answering that you would run the evaluation suite against the successor before switching, and hold the comparison rather than accepting the migration blind, is what an operator says.

You will be asked how a change gets to production safely. The answer that distinguishes operators is a versioned evaluation set running automatically on every change to prompts, retrieval, context assembly or model version — with per-case results, not just an average.

The follow-up worth preparing: a change improves the average and breaks three previously passing cases. Do you ship it? The right instinct is that averages hide regressions, and the per-case view is what you look at.

Add that you would look at which three broke before deciding. If they are edge cases nobody hits, shipping may be right; if they are the refusal cases where the system should say it does not know, the average improved while the product got worse in the way users notice.

• What triggers an evaluation run, and what blocks a deploy?
• How do you catch a provider-side behaviour change?
• How do you grade outputs that have no single correct answer?
• How often do you calibrate a model grader against humans?

Classical ML concentrates cost in training. LLM systems concentrate it in serving, where it scales with usage and hides inside a per-request token count nobody looks at until the invoice arrives.

Be ready to reason out loud: what drives cost per request, which changes move it, how you would cut it without hurting quality, and how you decide whether a two-point accuracy gain justifies doubling spend. That last one is a judgement question, and "it depends on the use case" is only a good answer if you then say what it depends on.

Have two concrete levers ready. Routing easy requests to a smaller model and caching a stable prompt prefix are the ones that most often produce large savings with no quality cost, and naming them specifically beats listing optimisation techniques abstractly.

Expect: "users report the assistant got worse this week. Nothing was deployed. What do you do?" This is the best question in the interview because it cannot be answered from theory.

Good answers establish facts before theorising — what changed, when it was last known good, whether the evaluation set shows it — then work outward to provider changes, data drift in the retrieval corpus, or a silently failing index. Weak answers jump to a suspected cause, which is exactly what makes real incidents last for days.

The detail that impresses is naming the failure nothing alerts on: a scheduled reindex that stopped succeeding, so new documents are written and never embedded. There is no error, no latency change and no cost change — only answers that quietly stop covering recent material.

Ship one small LLM feature and operate it for a few weeks. Add an evaluation set, track cost per request, and let something break. That gives you concrete answers to every question above.

If you come from MLOps or platform engineering, lead with that experience rather than apologising for it. Deployment, monitoring, rollback and reproducibility all transfer; what you add is evaluation design and token economics, and interviewers know the gap is narrower than job descriptions imply.

Frame the transferable parts in their vocabulary. Saying that you would treat the prompt and the retrieval index as deployable artefacts with versions and rollback is the same instinct you already have, expressed in terms that make the fit obvious.