LLM operations: from demo to a run someone owns

A prototype needs an API key and an endpoint. That is enough for a demo. For operations you need: who is allowed to change the prompt, what a rollback looks like, where the secrets live, and who picks up the phone at night when the model starts answering nonsense.

I watch this tip over at the same point in almost every project. The demo convinces, the prototype goes live "for now," and six weeks later nobody has an overview of which prompt version is even running.

Prompts are code

A prompt sitting in a chat window at the provider, or in a config table with no history, is not versioned. It is a state nobody can reproduce.

What I set up instead:

Prompt files in the repo, with commit history like any other code

A version number carried in the log of every run

Review before merge, the same as for a code change

That sounds small. The difference shows up the moment an answer goes wrong and someone asks which prompt was running at the time. Without versioning, the answer is a shrug.

The pipeline, not the single call

An LLM call is never alone. Before it: retrieval, filtering, assembling the context. After it: parsing, validation, sometimes a second call to check the first. Every step can break, independent of the model.

For an MCP server that gives a chatbot tools, this means concretely: the server has to run, has to be redeployable without taking the chatbot down with it, and someone has to know who restarts it during the next incident. A model update on the provider side must not mean retesting the whole pipeline because nobody knows which version is running where.

Secrets that do not end up in the default repo path

API keys in a .env that ends up in the repo by accident. A key shared across several projects because creating a new one was too much friction. Rotation that is theoretically possible but never happens in practice, because nobody knows where else the key is used.

That is not an AI-specific problem, but LLM projects make it worse. A key is often tied to a billing account with a real limit. If it leaks, the damage is not only a data breach, it is an invoice.

A rollback, walked through

A model update on the provider side changes response behavior. The error rate in evaluation climbs. Without a prepared rollback path that means someone has to figure out under time pressure which prompt version ran before the update, whether it is still compatible with the current retrieval setup, and how to route traffic back to it without tearing apart running sessions.

With a prepared path, that is a deploy with an old tag, not an emergency. The difference is not made during the incident, it is made weeks earlier, when someone decided a rollback is a tested procedure and not an improvisation.

Who owns the run

The question that stays unanswered most often in reviews: if the system starts giving wrong answers at 3am, who gets the alert, and what is that person allowed to decide alone? Roll back to the last prompt version? Turn the model off? Notify users?

Without an answer to that question, operations is a prototype carrying production load. That holds until it does not, usually on a Friday evening.

In small teams the answer is often "whoever built it." That works as long as that person is reachable and stays the only one who understands the prototype. The moment a second system joins, or that person is on vacation, "I know how this works" stops being an operating model and becomes a named single point of failure.

The pipeline for the pipeline

A prompt change should run through the same CI as a code change: evaluation set against the new version, diff against the old one, then merge. That sounds like overhead for "just a piece of text," but it is exactly where most regressions come from. A prompt that reads better in one session can quietly get worse in a different context, and without an automated comparison nobody notices before the user does.

Where I set this up, the evaluation set runs as its own CI job that fires before every merge on a prompt or retrieval branch. Red means no merge, no matter how good the demo looked in the team chat.

What I check for in a review

Is the prompt versioned in the repo, with history?

Can a run be traced to a version in the log, not just from memory?

Is there a rollback path that runs in under an hour?

Does one named person know they own the incident?

This ties closely to observability: without version logs an incident cannot be reconstructed. And to evaluation: a rollback without a set that measures the old version against the new one is a jump in the dark.

What the readiness check does with it

Day 4 of the LLM readiness check asks exactly these questions: prompt versioning, rollback, ownership. "We'll figure that out once it's live" counts as a finding, not a plan. A prototype without these answers is not a go-live, no matter how good the demo was.

The service behind this sits under AI engineering. Related: reconstructing answers, evaluation in production.