Tactical RAG Fails on the Index, Not the Latency

Most writing about RAG at the tactical edge is about latency. Can a vector search run on a vehicle computer fast enough to matter.

It can. A few hundred thousand embeddings fit in memory on modest hardware, and a nearest-neighbor query against them takes milliseconds. Speed is not the hard problem.

The hard problem is that a cloud retrieval system assumes a corpus that is complete and current, and at the edge the corpus is a shipment. Somebody decided what to load before the unit left. That decision, made weeks earlier by someone who is not there, sets the ceiling on every answer the system gives.

What actually breaks

Three things, and none of them is throughput.

Coverage. The index contains what was packaged. If the relevant technical manual was updated after the load, or the threat report from last week never made it onto the drive, the system will retrieve the second-best passage and answer confidently from it. A retrieval system does not know what it does not have. It returns the closest thing in the index, and the model treats that as context.

Version skew across the fleet. Two vehicles with two different index snapshots answer the same question differently. Nobody notices until an after-action review compares them. If the system is making recommendations, a divergent answer is worse than no answer, because both crews believe they have the authoritative one.

The embedding model. The index is only valid for the model that produced it. Change the embedding model without re-embedding the corpus and every query returns plausible garbage, silently. This is the failure I would bet on in the field, because swapping a model file is a one-line change that looks like a minor upgrade.

Three deployment patterns

The patterns are not complicated. The tradeoffs are about who owns a failure.

Embedded in the application. The index ships as a file next to the binary, opened in process. Nothing to administer, one security boundary, and a crash of the process takes retrieval with it. Good for a single-application system with a fixed corpus. Bad when two applications on the same platform both want the same index and neither wants to own it.

A local service. The index runs behind a small API on the platform, with its own resource limits and its own logs. More moving parts, better isolation, and it gives you one place to record queries, which matters more than the performance. The cost is a network surface on a platform that may already have too many.

Hybrid, with opportunistic sync. A local index plus a mechanism to push and pull updates when a link exists. This is the pattern everyone wants and the one that produces the most interesting failures, because sync is where the corpus model breaks.

Sync is a conflict problem wearing a networking costume

The naive design is last-write-wins. Newer document replaces older, ties broken by timestamp. That is correct for a wiki and wrong for intelligence.

Two units can hold two documents about the same bridge, both true, both from different sources, with different confidence. Collapsing them into one row destroys the thing an analyst needs: which source said what, and when. The vector store should hold both, with provenance attached to each, and the retrieval layer should surface the disagreement rather than average it away.

So the metadata matters more than the embedding. Every vector needs, at minimum: the source identifier, the collection time, the classification or handling caveat, the index build identifier, and the embedding model version. Without those five fields you cannot answer the only question that matters after a bad decision, which is where the retrieved text came from and whether it was current.

One more mechanic worth stating plainly: deletions are harder than inserts. A document pulled for being wrong has to be removed from every index that holds it, including the one on a vehicle that has been offline for a month. If your sync protocol cannot express a tombstone, you will ship a system that keeps quoting a retracted report.

What to prove before it ships

Treat the index like any other configuration item, because that is what it is.

Build it from a known corpus snapshot and record the hash. Pin the embedding model by version and refuse to open an index built by a different one. Refuse to start if the index is older than a threshold you set, or start in a degraded mode that says so on screen.

Then test retrieval, not the model. Write fifty questions whose answers you know, with the passage that should be retrieved. Run them against the packaged index. You are measuring recall on your own corpus, which is the number that predicts whether the system is useful, and it takes an afternoon.

After that, test the degrade path. Pull the network, age the index, corrupt one shard. The system should get worse in a way the operator can see. A retrieval system that fails silently is more dangerous than one that fails loudly, because the operator keeps trusting it.

The part that is not technical

The packaging decision is a command decision. Someone has to choose what goes on the drive, and that choice is a bet about what the mission will need. The system should make that bet visible: what is loaded, how old it is, and what was left out.

A fielded retrieval system is a claim about the corpus. If nobody can say what is in it, the claim is unverified, and the crew is trusting a search result they cannot audit.

That is the design work. The vector database is the easy part, and it will be the part you replace first.