The most interesting AI app companies we’ve met this year don’t look like typical app companies. They train their own models, decide which model handles each step of the work, and build the plumbing that keeps their agents running.
In other words, they look more like infra companies.
That cuts against the usual story, in which the labs keep moving up the stack and apps get squeezed into a thin layer on top. The labs are certainly coming for part of the app layer. Where a smarter model makes a better product, as in writing, coding, and research, they have every advantage: the model, distribution, capital, and brand. But the app layer isn’t one thing. Where the work depends on knowing a particular company’s processes, rules, and regulators, a smarter model alone won’t get it done.
That second kind of work is where startups can win, but how they win is changing. In the first wave of AI apps, a startup’s edge was mostly at the surface: a well-designed workflow around a model API. Now more of the edge lies underneath.
We’ve seen this cycle before. In the late 1990s, the first SaaS companies racked their own servers and wrote their own management software. The cloud took that burden away, and for two decades a software company could live at the top of the stack and rent everything beneath it. AI has followed the same arc. The ML companies of the 2010s gathered their own data, trained their own models, and built their own infrastructure for training and serving them. Then model APIs arrived, and for a couple of years an app could be a good workflow around a rented model.
Now the pendulum is swinging back. When apps are easy to build, they’re also easy to copy. This part of the cycle rewards technical depth.
What’s changed
Over the past year or so, four things have changed in startups’ favor:
Apps can now own their models.
Open-weight models now come close to the best closed models, and a startup can train them further on its own domain. Hippocratic, which builds AI agents for healthcare, runs its product on 30+ open-weight models. Its co-founder and CEO, my friend Munjal Shah, explained on the B2BaCEO podcast that Hippocratic charges around $10 an hour for its product; running it on OpenAI’s models would cost more than $100 an hour. At that cost, the business wouldn’t work.
Owning models matters for quality, too. AI is still uneven, brilliant at some tasks and unreliable at others. Munjal described one of Hippocratic’s agents asking a patient whether she’d prefer her Pap smear in person or remotely. Models improve fastest on what benchmarks measure, so gaps like that don’t close reliably from one release to the next. A startup that owns its models can find and fix them.
Agents made apps more complex, and more expensive to run.
An agent turns one request into 100s or 1,000s of steps, and each step costs money. Even as the price of tokens falls, the cost of finishing a job is rising. In traditional software, serving one more customer cost almost nothing, so an app’s architecture mainly determined how well it worked. Now it determines both how well the work gets done and what it costs.
Routing is part of it: sending each step to a “good enough” model, and calling on the best models only when necessary. Good architecture also means handling rule-based steps with ordinary code, giving each model call only the information it needs, reusing work instead of repeating it, and checking each step’s output so mistakes are caught before they snowball. Getting these choices right takes detailed knowledge of the customer and their workflows, which a vertical startup has.
Security, governance, and data privacy are now top concerns.
AI raises questions buyers didn’t have to ask about earlier software: what an agent can do on its own, whether anyone can reconstruct what it did, and if their data is being used for training. The right answers depend on the field, and a vertical startup can design its product around them from the start. Hippocratic decided early that its agents would stay within a nurse’s scope. They don’t diagnose or prescribe, and they don’t handle cases like high-risk pregnancies at all.
Trust is increasingly what customers are buying.
For someone buying software at a large company, the downside of a bad decision is greater than the upside of a good one. A general-purpose model can be used for almost anything, so no one can test it against everything it might be asked to do. A vertical startup can test its agents on the situations they’re most likely to face and measure them on the outcomes its customers care about.
Hippocratic spent 18 months testing before launch. Thousands of licensed nurses called the agents posing as patients and logged every error, and the same test was run on human nurses for comparison. Hippocratic only shipped once its agents made fewer mistakes than the nurses did. No public benchmark could have measured that. Hippocratic also accepts liability if its product harms a patient. Customers want a vendor willing to take that responsibility, and the labs are unlikely to do so for every industry at once.
Insights for technical founders on the path to CEO.
Learn how to build and scale an enduring company from Ashu and leading founder-CEOs.
Best practices for building AI apps
From the founders we work with, here are a few principles for what makes a strong app company:
It sells outcomes. The strongest founders start from an outcome a customer cares about and work backward. The best can price on that outcome, charging only when the work gets done. They’re judged by their customers’ results, not by public benchmark scores.
It picks a vertical where mistakes are costly. In finance, law, and healthcare, errors can be very expensive, so even a small edge in accuracy can win a customer. As Munjal puts it, run into the vertical as fast as you can.
It runs an organization’s workflows, not one person’s. Most AI products today, including much of what the labs sell, help one person do their job faster. That rarely adds up to a more productive company. Individual tools also keep what they learn with one person. An app that runs work across a company improves the whole process, and what it learns compounds.
It has the technical depth of an infra company. Early in a platform shift, much of what the platform needs doesn’t exist yet, so the best teams build it themselves. When we backed TubeMogul, AWS was new, and running a complex advertising app on it took three technical founders. Agents are at that stage now.
Most importantly, it’s where the work happens, and it gets better from doing it. A tool sits on top of work that lives somewhere else, and it’s easy to replace. An app the work runs through isn’t. Because the work happens there, the app can gather trace data: a record of each step and decision, including the exceptions. Over time, that record becomes what we call a context graph. Along with the app’s own evals for what good work looks like, it makes the app better the longer it runs, and harder to replace.
Always audit-ready: How two IIT grads are teaching AI to run finance
Finance is exactly the kind of vertical where AI apps can win. Mistakes are extremely costly, the work is highly regulated, and success is clearly defined: the books either reconcile or they don’t.
Our portfolio company Hyphenate is building agents that do the work of a full finance team, from invoicing to the month-end close. We’ve backed the company since its seed round, and this week it announced a $45M Series A. To mark the occasion, we sat down with its founders, Ram and Ajay, to learn how they built it.
