Eighteen Months, One Phase One, No Product

Eighteen months ago I stopped optimizing for what was new and started optimizing for what would survive a review.

That is a dull sentence to open with, and it describes the most useful decision I have made in this field. The work was not a product launch. It was architecture, written notes, and a lot of time spent on the parts that do not demo.

One thing did happen that is worth reporting. A research proposal I wrote was selected for funding as a Phase I SBIR, funded through the Department of Defense's Chief Digital and Artificial Intelligence Office and administered by the Department of the Air Force's Digital Transformation Office. The award notice carries the details.

The long, illustrated version of this story is the Long Game. This note is the argument without the scenery.

I want to be precise about what that means, because the temptation is to present it as validation of a product. It is not. Phase One buys a feasibility study. It says a group of people who review proposals for a living thought the question was worth a small amount of money to answer. That is a real signal and a small one, and confusing the two is how founders end up with a company that has awards and no customers.

The noise was not the problem

Every few weeks, a new model. A new framework. A new approach that supposedly changes the premise.

The interesting thing about that cycle is not that it is distracting. It is that it rewards a specific behavior: having an opinion quickly. The reward is social, and it is disconnected from whether anything gets built.

I watched people with more talent than me spend a year becoming fluent in the releases and finish with nothing that could be deployed. I also watched people ignore the releases entirely and build things that were obsolete by the time they shipped. Both failure modes are real, and the difference between them is not attention. It is filter.

Four questions

I ran everything through four questions. They are not clever. They are ordered, which is the only thing that makes a filter useful, because the first one eliminates most candidates in a minute.

Does it solve a real problem, or a technical one? Most releases solve a technical problem. That is fine for a lab and useless for a program office. The test is whether you can name the person whose week gets better.

Can it work in a zero-trust environment? Not "does it support SSO." Can it run where every request is authenticated and authorized, where the network is hostile by assumption, and where the audit log is not optional. A surprising amount of the ecosystem fails this before you get to the interesting part.

Can someone verify the output? Not "is it accurate." Can a second party, later, establish why the system produced this result. If the answer is a log line that says a tool ran, the answer is no.

Does it work across security levels? In federal work, the same capability usually has to exist at more than one classification, and the version that only works at the lowest one is a different product with a different deployment story.

Most of the time the answer was: interesting, not for this. Saying that forty times is the actual discipline. It is also what makes the four or five times you say yes credible.

What the work actually was

The hard part was never the model. It was decision infrastructure: taking messy institutional data and turning it into something a leader can act on and later defend.

That work looks like months of architecture before the first line of operational code. Looking back, the order was right and the amount of time was not, because I re-derived the same three questions in every design review.

What does the decision need, in what form, and what does it leave behind. Those questions produced a design that is verifiable before it is capable, and that runs inside a boundary rather than beside one. Designing for verifiability instead of benchmark scores is not a moral position. It is a bet that the buyer will eventually ask how the system knows, and that the vendor who can answer will still be in the room.

The model is the part that changes. The record of what it did is the part that has to hold.

What the award is and is not

It is evidence that a reviewer agreed the problem is real and the approach is plausible.

It is not evidence that the approach works at scale, that anyone will buy it, or that the transition path exists. Those are Phase Two questions and customer questions, and a Phase One award tells you nothing about either. Anyone who presents an SBIR as market validation is skipping a step that takes years.

What it did change is narrower and worth having: it made the work legible to people who fund things. A proposal with a funded predecessor gets read differently than an unsolicited idea, which is an uncomfortable fact about how institutions evaluate novelty.

What I would tell someone starting

Two things, and neither is about AI.

Write the decision down before you write the code. Who decides, what they look at, what they would accept instead, and what record the decision leaves. If you cannot fill in those four, you are building a demo and calling it a product.

Then spend your time on the parts that survive the model being replaced. The integration, the boundary, the evidence, the retraining plan, the person who owns it in eighteen months. None of that appears in a launch post, and all of it decides whether the thing is still running.

The eighteen months were worth it. Not because of the award, which is a small thing, but because the filter got sharper. I can now tell, in about ten minutes, whether a new capability changes what I can build for a customer who has to defend the result.

That is the skill I was actually acquiring. Everything else was noise with good marketing.

If you want the longer version, with the hype timeline and the full account of the four questions, it is here.