How can a venture fund tell whether its portfolio is getting faster with AI?
3 min read
›Contents
The short answer: read every company under the same rules, from a baseline frozen before anything changes, and combine the results by default so no company is named. Share a company's own numbers with the fund only if that company opts in, in writing. Numbers the companies report about themselves cannot be compared with each other, and a view that exposes a company by default is one that company has little reason to join.
This is the problem Kelright's Benchmark is built for when it is run across a portfolio. It is written for the head of platform, or the partner who has to say what the fund's AI work has done for the companies it backs.
Why self-reported numbers do not compare#
Ask ten portfolio companies how much faster AI has made them and you are likely to get ten answers measured ten ways: one counts deployments, another story points, another the share of engineers with a licence. Even where two use the same word, "lead time" can start at a commit, a ticket or a request.
Without a shared definition there is no comparison, only a spread of opinions. The fix is not a better survey. It is reading every company from its own systems, under one set of rules.
Same rules, frozen before#
Two conditions make a portfolio readable as a cohort:
- One set of rules. Every value is computed under a named version of the same rules, so a company's lead time and another's mean the same thing.
- A before that cannot move. Each company's baseline is read from history when it connects, frozen once, and never recomputed. "Faster than before" is then a statement about the same before every time you look. How the baseline works.
Together they let a fund read the cohort quarter after quarter without asking whether the yardstick changed.
Combined by default, named only by consent#
A fund reads a cohort. It is not entitled to a company's private numbers. So the cohort view is built to protect them:
- the fund sees the cohort combined, with no company named and no person among the data;
- a company's own numbers reach the fund only if that company opts in, in writing;
- each company decides for itself, including whether to take part in the Guild as a cohort.
A view that is private by default is one a company can say yes to. A view that exposes it by default gives it little reason to.
Which measures a fund should read#
A fund does not need forty measures. It needs the few that show whether agents are making companies faster or only making them spend more:
- the four DORA measures, for delivery;
- agent-assisted pull request share, for how much of the work agents do;
- AI spend per merged pull request, for what it costs;
- rework rate, for whether agent work holds.
What to ask of any tool that offers a portfolio view#
Four questions separate tools that report a portfolio from tools that can be trusted to:
- Is the baseline frozen, or can it be recomputed?
- Are the same rules applied to every company, and are they versioned?
- Does the fund see companies by default, or only combined?
- Does a company's data leave its own systems?
Kelright's answers are on the comparison page, beside the same questions for any other vendor.
What the companies get#
A portfolio reading that works for the fund has to work for the company first. Each company gets its own reading of its own delivery, in its own Datadog, with a baseline it can trust and a way to see whether what it is doing is working. The cohort is the sum of companies that each got something.
If you back software companies and want to read them this way, write to us with the fund and the companies you would start with.