Inteltech
AI Sustainment Is Performance Sustainment
The RDT&E to O&M shift exposes a gap: O&M contracts are scoped for stable systems, and an AI system is never stable enough to leave alone.
The budget account changes. The system does not become stable.
When an AI program moves from research, development, test, and evaluation to operations and maintenance, the milestone is treated as a finish line. The system works. Now keep it running. That framing fits an aircraft or a radio. It does not fit a model whose performance depends on data that keeps changing.
The O&M contract is usually scoped at initial operational capability. It pays for uptime, patches, and a help desk. An AI system needs those things and three more: data stewardship, model evaluation, and the people who can investigate a performance drop. A contract that does not fund those lines will underfund the system within a year.
What the account change assumes
O&M assumes a defined configuration. The system has a baseline, a maintenance schedule, spare parts, and a requirement that is expected to remain stable. The job is to keep the known system inside its envelope.
An AI system has a baseline too, and the baseline moves. The data distribution shifts. The upstream schema changes. The model is retrained. A new sensor or a new policy changes the inputs. The envelope is not fixed, so the maintenance model has to include change.
That is why the RDT&E to O&M transition is a useful forcing function. It makes the program office write down what it will pay for after the prototype team leaves. The answer is often a surprise.
Four sustainment lines
Data. The pipeline that feeds the model is a production system. It needs schema monitoring, quality checks, lineage, and an owner. When an upstream system changes a field, the model does not throw an error. It produces a quieter, worse answer. Data stewardship is how you catch that.
Model. Retraining is not a patch. It requires a labeled set, a training run, an evaluation against a baseline, a deployment, and a rollback. Each step has a cost and a failure mode. A model registry and a versioned evaluation set are the minimum.
Monitoring. Uptime is not performance. A system can be available and wrong. Monitor the task metric, the refusal rate, the latency, the cost per inference, and the input distribution. A drift threshold should trigger a review, not a ticket that sits in a queue.
People. Someone has to be able to investigate a bad output. That person needs access to the data, the model, and the evaluation set. A traditional system administrator can keep the server up. They cannot tell you why the model's recall dropped on a class of documents.
The O&M contract should fund those four lines explicitly, not assume they fit inside a percentage of development cost.
Why the handoff fails
The team that built the model is often not the team that sustains it. The R&D contractor delivers a model, a slide deck, and a repository. The O&M contractor inherits a system it did not design.
The documentation is incomplete. The training pipeline is a notebook with hard-coded paths. The evaluation set is a folder on someone's laptop. The new team optimizes for the metrics in the contract, which are usually uptime and ticket closure, not task performance.
I have seen this pattern in federal programs. The fix is not a longer handoff meeting. It is a transition package that is a deliverable with acceptance criteria: the data dictionary, the training pipeline, the evaluation set, the model card, the runbook, and a list of known failure modes. If the package is missing, the government should not accept the transition.
What to write into the contract
A performance baseline and a re-evaluation cadence. The baseline should be the same metric used at acceptance, measured on a held-out set. The cadence should be frequent enough to catch drift before a user does.
A data quality agreement. Who owns the upstream feed? What happens when the schema changes? How quickly must a break be reported?
A model update process. Who approves a retrain? What evidence is required? Can the government freeze a version? What is the rollback path?
A knowledge transfer milestone. The transition package is not a courtesy. It is the thing that lets a second team run the system.
A compute budget. Inference and retraining are recurring costs. A program that funds the model development and not the compute will watch the system degrade or stop.
Four things to avoid.
Do not treat the model as a finished product. It is a component with a lifecycle.
Do not buy O&M as a percentage of development cost without scoping the work. The percentage is a convention, not an estimate.
Do not assume a vendor's model update is a free improvement. It is a change to a production system, and it needs a test.
Do not measure uptime as a proxy for performance. The system can be up and wrong.
The finish line that is not one
The RDT&E to O&M shift is not the end of the project. It is the point where the system starts to change without you. The contract should assume that, not resist it.
A program that funds data, model, monitoring, and people after IOC is buying the ability to keep the system honest. A program that funds only uptime is buying a server that happens to run a model.