The terms of agentic engineering, defined the way the numbers are made.
What agentic engineering is, what the four DORA measures and the agent-native measures mean, and how Kelright's Benchmark computes each one. Every definition starts with the term and stands on its own.
Take the readiness check
Prefer a link by email?hello@kelright.com
The practice
What it is to build with agents, and what the Guild installs.
- Acceptance check
- An acceptance check is the test, read from the customer's own systems, that says a Guild milestone is done: coverage reported in CI, pipeline time, a ticket register closed, or the share of pull requests agents helped write. A milestone is not accepted on a slide or on Kelright's word.
- The Guild's milestones
- Agentic engineering
- Agentic engineering is building software with AI coding agents that take on real work, such as writing, testing and reviewing changes, while the team's own tests, gates and review rules decide what merges. The checks go up before the agents do.
- The Agentic Guild
- AI coding agent
- An AI coding agent is software that takes a task, such as an issue or a described change, and works on it in a team's repository with some independence, opening pull requests that people review. It differs from an autocomplete assistant, which suggests code as a person types.
- Champions
- Champions are the two of a team's own engineers that the Guild trains to run and extend the guardrails after the handoff, so the practice does not depend on Kelright staying.
- What stays after the Guild
- Diagnostic
- The diagnostic is the Guild's first milestone, M0, in weeks zero to two: Kelright reads a team's repositories, pipeline and tracker, measures where it starts, and freezes the baseline in the Benchmark before anything changes.
- M0 and the other milestones
- Gate
- A gate is a check in CI or in review that a change must pass before it merges, such as a coverage floor, a required test on an agent-written change, or a limit on change size. A gate holds agent-written code to a team's tests without a person watching every change.
- Guardrail
- A guardrail is software that tells engineers and agents what to do at the moment they do it: agent instruction files that carry a team's conventions, CI gates, and the branch protection and review rules that match. Guardrails live in the team's own repository and keep running after Kelright hands off.
- What the Guild leaves running
Readiness
Where a team stands before it starts.
- Agentic Readiness Check
- The Agentic Readiness Check is Kelright's short assessment of how ready an engineering team is to work with coding agents: about six minutes, fourteen questions after a few lines about the company, no account, and a written read of the team's phase and the three places to look first.
- What the check asks and gives back
- Output outrunning the checks
- Output outrunning the checks is where a team lands when agents went up before the checks did: more code is produced than the tests, gates and reviewers can verify, and review is where the work piles up. The fix is to tighten the gates, then hand agents more.
- Phase
- A phase is one of the four stages the readiness check places a team in as it climbs: fundamentals first, a sound house ready for agents, agents in use with a couple of things behind, and already compounding. The check says which phase a team is in and what to do to climb to the next.
- The phases a company climbs
What the Benchmark measures
The measures, as the Benchmark computes them.
- Agent-native measures
- Agent-native measures are the engineering measures that only exist once AI agents write code, such as agent-assisted pull request share, AI spend per merged pull request, and cycle time and rework compared between agent-assisted work and other work. The Benchmark reads them in its Agent adoption and Agent economics families.
- Agent-native measures, answered
- AI spend per merged pull request
- AI spend per merged pull request is what agent work costs for each pull request that ships: the agent cost in a window divided by the pull requests merged in it. It measures spend that lands, not spend per line written.
- Spend that lands
- Change failure rate
- Change failure rate is the share of production deployments that fail. The Benchmark computes it as the share of successful production deployments that were rolled back or linked to an incident, and lower is better. In DORA's current guidance it is one of the two instability measures, with deployment rework rate.
- Deployment frequency
- Deployment frequency is how often a team deploys to production. The Benchmark computes it as successful production deployments per day over a window, and higher is better. It is one of DORA's throughput measures.
- DORA metrics
- DORA metrics are the software delivery measures from the DevOps Research and Assessment program at Google Cloud. Its earlier research named four: deployment frequency, lead time for changes, change failure rate and time to restore service. Its current guidance names five, in two groups: throughput (change lead time, deployment frequency and failed deployment recovery time) and instability (change fail rate and deployment rework rate). The Benchmark's Delivery family reads the four earlier measures.
- DORA's own guide to the metrics
- Lead time for changes
- Lead time for changes is how long a change takes to reach production. The Benchmark computes it as the median time from a commit being authored to its first successful production deployment, and lower is better. It is one of DORA's throughput measures.
- Pull request cycle time
- Pull request cycle time is the median time from a merged pull request's first commit to its merge, and lower is better. The Benchmark also splits it into time to first review, review time and merge time, to show where a pull request waits.
- Rework rate
- Rework rate is the share of pull requests merged in a window that were since reopened or reverted, and lower is better. The Benchmark reads it for agent-assisted and other pull requests separately, so a team can see whether agent work comes back more often. It is not DORA's deployment rework rate, which counts unplanned deployments that follow a production incident.
- Tests on agent-written pull requests
- Tests on agent-written pull requests is the share of merged pull requests credited to an agent that changed a test file, and higher is better. It is one way the Benchmark reads whether agent work is verified before it merges.
- Time to restore
- Time to restore is how long a team takes to recover when something breaks in production. The Benchmark computes it as the median time from an incident being opened to its being resolved, and lower is better. It is the recovery measure of DORA's earlier four, time to restore service. DORA's current guidance replaces it with failed deployment recovery time, which measures recovery from a deployment that failed and needed immediate intervention, and counts it with throughput. The Benchmark's measure is incident-based and is not that one.
How the Benchmark keeps its word
The rules that make a before and an after mean something.
- Cohort view
- A cohort view is how a venture fund reads the companies it backs as one group under the same rules: combined by default, with no company named, and with a company's own numbers reaching the fund only if that company opts in, in writing.
- How a fund reads its portfolio
- Frozen baseline
- A frozen baseline is the before the Benchmark measures every change against: ninety days of history read when a team connects, frozen once every connected system has been read, and never recomputed afterwards, by the customer or by Kelright. It makes an after a claim that can be checked.
- Before and after
- Rule version
- A rule version is the name of the set of rules a Benchmark value was computed under. An after is always read under the same rule version as the before it is compared with, and when the rules change the original window is measured again as a second before beside the first.
The category
Where Kelright sits among the tools that measure and check engineering.
- Engineering intelligence platform
- An engineering intelligence platform is software that reads the systems engineering teams already use, such as source control, CI and trackers, to report on how a team delivers. The Kelright Benchmark is one, designed around a baseline that cannot be recomputed and a cohort view for the funds that back software companies.
- How to choose between them
- Verification of agent work
- Verification of agent work is checking that what an AI agent produced is right, not only that it was permitted or that it runs. Tests check code against what it was written to do; a decision record checks the work against what the company decided. Kelright's Guild installs the first kind of check and the Brain provides the second.
- The Brain
The Brain
How a company's decisions become something an agent can be checked against.
- Decision record
- A decision record is the Brain's cited record of what a company decided: each value carries the passage it came from with the part that matters marked, how it was settled (a person approves whatever the Brain is unsure of), and the page in the company's own tools it was written to. Agent work is checked against it before the agent acts.
- The record
- Held change
- A held change is a change an agent wrote that passes every test and is still stopped, because a decision the company's people approved says it needs something more, such as a second reviewer. The Brain names the decision and the words it rests on, so whoever reviews the change knows why.
- The check