Intelai
The Open Stack's Advantage Is Provenance
Cost is the least interesting reason to build on open components. The one that survives procurement is knowing where the weights and data live.
The first version of an open stack usually gets argued about at the wrong layer. People debate vector databases for a week and never ask the question that decides whether the system can be deployed.
Where does the model run, where does the data sit, and who can prove it.
In federal work those three answers are the product. Everything else is a preference.
The advantage is provenance, not price
The usual pitch for open source is cost. No per seat license, no token markup, no procurement for a new tool. That is real, and it is the least durable advantage, because vendors can cut prices and open source is not free once you count the engineering time.
The durable advantage is that you can answer questions a closed system cannot.
Which model weights are running, and where did they come from. What data was used to fine tune them. Where does the prompt go when a user submits it. Where is the log, and can I export it. If the vendor disappears, what is left.
Those questions come up in every assessment I have seen. A closed system can answer them with a contract and a trust relationship. An open one answers them with an artifact you can inspect, which is a different kind of answer and usually a shorter conversation.
With open components you are buying an exit, not a license.
Choose by exit cost
Three tests, applied to every layer.
Can it run with no outbound network. Not every component will, and the ones that will not need to be identified before an air-gapped deployment, not during one.
Can you replace it without rewriting the system. A vector database behind an interface is a component. A vector database whose query syntax leaked into forty files is a dependency.
Can you explain what it does to a reviewer who is not an engineer. If the answer requires a diagram of a managed service you do not control, the layer is a liability in a regulated environment.
The stack, layer by layer
| Layer | Reasonable choices | What the layer is for | What swapping costs |
|---|---|---|---|
| Interface | Next.js, Streamlit | A review surface for humans | Low, if the API is separate |
| API | FastAPI | The contract every client uses | Low, and worth owning |
| Orchestration | LangGraph, plain code | Control flow, retries, state | Medium, and the highest churn |
| Retrieval | PGVector, Weaviate, FAISS | Finding the right context | Medium, if the interface is clean |
| Embeddings | Open embedding models | Turning text into vectors | High, because it requires reindexing |
| Models | Open weight models | Generation and reasoning | Medium, if prompts are versioned |
| Serving | vLLM, Ollama | Throughput and hardware fit | Low, if the API is OpenAI shaped |
| Storage | Postgres, DuckDB, Redis | Structured data, analytics, cache | Low to medium |
| Security | Vault, Keycloak, OPA | Secrets, identity, policy | High, and it should be |
Read the last column as a budget. The layers with high swap cost are where you should spend the design time, because those are the decisions you will live with.
Embeddings are the one people underestimate. Changing the embedding model means reindexing the corpus and re-running evaluation, and if the index is large that is a project rather than a deploy.
Where open source loses
Three places, and they are real.
You are the support contract. When the retrieval quality drops at 2am, there is no ticket to open. There is a maintainer in a different time zone and a GitHub issue, and your team is the one that has to understand the code well enough to fix it.
Evaluation is on you. A vendor ships a benchmark and a dashboard. With open components you get a model and a blank page. If you do not build an evaluation set from your own documents, you will not know whether the system works, and the absence of a vendor claim is not evidence of quality.
Patching is on you. A vulnerability in a dependency is your problem on your schedule. Closed systems have the same problem and hide it better, which is not obviously an improvement, but it does mean the work lands on someone else.
None of those are reasons to avoid the stack. They are reasons to staff it. An open stack with no one assigned to evaluation and patching is a science project with a production URL.
Build order that actually works
Start with retrieval and evaluation, not with the model.
Assemble a corpus of real documents, a hundred questions with known answers, and a retrieval pipeline. Measure whether the right passage comes back. That number limits everything downstream, and it is cheap to measure.
Then pick a model. Any reasonable open weight model will do at this stage, because the retrieval is doing most of the work and the model is doing the synthesis. Test two, keep the one that cites its sources better.
Then build the evidence layer. What gets logged, where, for how long. This is the layer that gets deferred and the layer that decides whether the system can be deployed at all.
Then the interface. Last, because the interface is the cheapest thing to change and the easiest thing to mistake for progress.
The stack will be different in a year. Some of these tools will be gone, some will be absorbed, and the orchestration layer will look nothing like it does now.
The requirement will not change. You will still need to say where the data went, which model touched it, and what record exists. A stack built to answer those questions survives a tool change. A stack built around a specific framework does not.
Choose boring components, own the interfaces between them, and write down the answers to the three questions before you write the first line of integration code.