Kelright
← All writing

How do you make sure AI agents build what your company decided?

3 min read

›Contents

The short answer: write decisions down as a record that cites the words they came from, have your own people approve it, and check agent work against it before the agent acts. When the work and the record disagree, hold the change and say which decision it contradicts. If the work was right and the record is stale, propose the update for a person to approve. Tests alone cannot do this, because the check that mattered was never written down.

Software can pass every check it has and still be wrong#

Tests say whether code does what it was written to do. They cannot say whether it was the right thing to write. The check that mattered, what your company decided, is spread across meetings, threads and documents that nobody turned into a rule.

When people did all the work, that was survivable. People were in the meetings. As agents take on more of the work, the missing check becomes the most expensive one, because the agent was not in the meeting.

What this is not#

Three things people reach for do not solve it:

  • Search finds text, but not which text was decided.
  • Permissions prove an action was allowed, not that it was right.
  • Evaluations grade models, not your own business's execution against your own ground truth.

What an agent's work needs to be checked against is your company's own decisions, each traced to its words and approved by a person.

Four steps, with a person at two of them#

This is how the Kelright Brain keeps agents correct.

  1. Capture. Meetings and documents become named decisions, commitments and owners, each with the sentence it came from. Nothing is a summary; each value keeps the passage it rests on.
  2. Approve. Your people see only what the Brain is unsure of, check it against the words, and approve or correct it in one screen. A person decides what the record says.
  3. Check. Agent work is checked against the approved record before it acts. A change that contradicts a decision is held, with the decision named.
  4. Correct. If the work was right and the record is out of date, the Brain proposes the change, and a person approves it.

Every value keeps its history: what it replaced, when, why, and on whose word.

What a held change looks like#

An agent opens a pull request to retry failed card charges in the billing service. Its tests pass and its coverage holds. It is still held, because a decision your platform lead approved says that anything an agent wrote in billing needs a second reviewer before it merges.

The check names the decision and the words it rests on, so whoever reviews the change knows why. That is a held change: it passes every test and still waits for a person. (This example is illustrative. It is made up to show the shape.)

Where the record lives#

A record nobody can read is not a record. The Brain reads your meetings in Zoom and your documents in Notion and Google Drive, and writes ordinary pages back into your own Notion or Drive, each value's source passage one click away. If you leave, the pages are still yours. A workspace lives in the United States or the European Union, chosen once when it is made.

How it fits with tests and gates#

The Brain does not replace tests or code review. Your tests ask whether the code works; the Brain asks whether it is what you decided. The two sit beside each other in the pipeline. Installing the first kind of check is what the Guild does: guardrails and CI gates that hold agent-written code to your tests. The Brain adds the second.

Where to start#

Pick one decision your engineers keep getting wrong, such as "agent-written changes to billing need a second reviewer", and write down who decided it, when, and the sentence it came from. That is the shape of everything the Brain keeps. To see how the Brain does it across your meetings and documents, sign in or request a workspace.

Get the next essay

New writing, straight to your inbox. No spam, unsubscribe anytime.