---
title: "Making It Trustworthy"
date: "2026-08-04"
description: "Part 2's factory opened its own pull requests but trusted every agent to be right. This is about the gates: an architecture reviewer that reads decision records, branch protection the fleet cannot bypass, and the human on the merge."
tags: ["engineering", "ai"]
series: "Software Factories"
part: 3
language: "en"
draft: false
---

Part 2 ended with a loop that opened its own pull request, and one fact: everything in it trusted the agents. The polecat wrote the code and its own test. The refinery rebased and opened the pull request without running a single check of its own. Nothing compared the change to the architecture. And when the fleet flagged that it had picked a language nobody specified, nothing was set up to do anything about it. It was a good demo. It would be a terrifying thing to happen in a production environment or anything slightly serious.

So this piece is about the distance between those two things. The excitement of watching work happen on its own, Part 2 already covered. What is left is the boring, load-bearing question underneath: how do you get to where you would let a fleet of agents near real code in a production environment? I spent some time and did some hardening on the same toy factory from Part 2, and decided to make it something you can trust. It is something you build, one gate at a time, and most of it is configuration you assemble yourself.

Same project throughout: `text-toolkit`, the little library of pure string functions from Part 2, with one architecture decision record that says every function is one file, pure, no dependencies, always tested. That single document is about to start doing real work. It is short, so here it is, straight out of `docs/decision-records/0001.ADR.md`:

```
# 0001. Structure of text-toolkit

## Decision
1. One function per file, under `src/`. The file name matches the function name.
2. Functions are pure: no side effects, no I/O, no global state.
3. No third-party dependencies. Standard library only.
4. Every function has one test file under `tests/`, covering its behaviour and edge cases.

## Consequences
A change that adds a dependency, spans multiple files, or ships without a test
is out of spec and should be rejected.
```

Four rules and a line about what to do when one is broken. Nothing in Part 2 read this file. That is what changes here.

## Building the gate

When you stand up a factory the way Part 2 did, you do not get a review system by default.

The refinery, out of the box, is a merge processor. It rebases a branch onto `main`, and if the rebase conflicts or a configured test fails, it bounces the work back to the coding pool. That is the whole of it. It does not read the diff to understand what the change is trying to do. Its own prompt forbids that. And it runs no tests at all unless you wire a test command in, which the default leaves empty. So the starting point is a fleet that will catch a merge conflict and nothing else.

Everything else people mean when they say these factories are safe, a real review loop, an architecture check, a human who owns the merge, a gate on the work before it starts, is assembled. None of it ships ready to flip on. I went in expecting to import a "hardening" pack and instead spent my time writing agents and formulas. That is not a complaint. But it is what it is: the base install is a starting point, the rest is on you.

Let's add the guardrails: something that reads a change against the decision record and rejects it when it drifts.

There is no built-in agent for this so I built one. In Gas City that means a new agent with a prompt that gives it a job, and a formula that says how the job runs. The reviewer's job: read every accepted decision record under `docs/decision-records`, read the diff on the branch, and decide pass or fail. It sits in the flow between the coder and the merge processor, so the path becomes coder, then reviewer, then refinery. On a pass it hands the change to the refinery to land. On a fail it does exactly what the refinery does with a bad rebase: it resets the work back to the coding pool with a reason attached and leaves the branch alone, so a fresh coder picks up where the last one left off and fixes only what was flagged.

Strictly, the reviewer never writes code, because the failure I was guarding against is a reviewer that quietly fixes the thing itself and launders the violation into the main branch. This is about building a gate.

And none of this is magic underneath. Its judgment is just a prompt, the same kind of prompt every other agent in the town runs on. Trimmed down, it says this:

```
# the reviewer's cardinal rule
You are an architecture reviewer, NOT a developer.
- You NEVER write application code. You read a diff and judge it
  against the decision records.
- On PASS you hand the bead to the refinery to land via pull request.
- On FAIL you bounce it back to the polecat pool with a rejection_reason
  and PRESERVE the branch. FORBIDDEN: fixing the code yourself, merging,
  or pushing.

# how it reads a change
git fetch --prune origin
BRANCH=$(bd show $WORK --json | jq -r '.[0].metadata.branch')
git diff origin/main...origin/$BRANCH     # the change under review
cat docs/decision-records/*.ADR.md        # the binding rules
```

That is the whole trick. It reads the rules and the diff, and it can say no. What makes it a gate is not intelligence, it is position: it stands between the code and the branch.

To test it I wrote a bead designed to misbehave. I asked for a `slugify` function whose stop-word list is read from a bundled JSON file, and I asked for a second helper, `titleCase`, in the same module. Both requests quietly break the ADR: reading a file is I/O in a function that is supposed to be pure, and a second function is a second file. A vague spec gets you a literal-minded agent walking straight into the trap, which is what I wanted.

Here is the bead, worded to lead a literal worker into both mistakes:

```
bd create --rig text-toolkit \
  --title "Add slugify string utility" \
  --description "Add a slugify(text) utility under src/. It should read its
stop-word list from a bundled JSON file in the repo, and also expose a
titleCase(text) helper in the same module file so callers can import both."
```

Writing the agent does not put it in the flow. Left alone, the polecat hands a finished branch straight to the refinery, exactly as it did in Part 2, and the reviewer sits there with nothing to do. It becomes a step only when the handoff points at it.

So I moved the handoff. The polecat's formula ends by reassigning the finished bead to whoever lands it. I wrote a variant of that formula, `mol-polecat-review-work`, and it changes one line: it hands off to the reviewer, not the refinery.

```
# mol-polecat-review-work, the last step: hand to the reviewer, not the refinery
bd update $WORK --status=open \
  --assignee="text-toolkit/gastown.reviewer" \
  --set-metadata gc.routed_to="text-toolkit/gastown.reviewer" \
  --set-metadata merge_strategy=mr
```

That reassignment is all of it. The bead lands on the reviewer, the reviewer passes it on to the refinery when it approves, and the path becomes coder, reviewer, refinery. I did have to register the reviewer with the city first, a few lines of config, so there was a pool for the bead to land on. After that, slinging the work with `--on` picks the formula:

```
gc sling text-toolkit/gastown.polecat tt-6whx --on mol-polecat-review-work
```

So, what happened then? Well, the polecat did what it was told. It wrote `slugify`, read the stop words from a JSON file with `require`, added `titleCase` in a second file, pushed the branch, and handed it to the reviewer.

![The polecat's first commit, the one that got bounced: five files for a one-function bead. slugify.js line 1 reads the stop-word list from slugify-stopwords.json with require() (I/O), and the commit bundles an unrelated title-case.js. Every violation the reviewer is about to name, sitting in the diff.](/images/gas-city-making-it-trustworthy/pr6-first-commit-violations.png)

As you can see, there are way too many things there according to our ADR. It did not pass our gate. On its first look, the reviewer wrote this back onto the bead:

```
Verdict: FAIL
Violated: ADR-0001 rule 2 (pure/no I/O); rule 1 + Consequences
          (one function per file / change spans multiple files)

1. Rule 2 (pure, no I/O): src/slugify.js line 1:
   const STOP_WORDS = new Set(require('./slugify-stopwords.json'));
   require() on a JSON file is a filesystem read at module load, I/O
   in a module that must be pure. The stop-words list must be an inline
   literal.
2. Rule 1 (one function per file): slugify-stopwords.json is a second,
   non-function file for the slugify unit, so slugify spans two files.
3. Same clause, independent instance: the commit also adds titleCase,
   unrelated to this bead. That function already exists independently on
   branch tt-4d4-titlecase, and landing it here would produce two divergent
   implementations under different filenames.
```

To my surprise, I did not set up anything regarding the third point. The reviewer noticed that the `titleCase` the polecat had thrown in was already being built on another branch, by another agent, under a slightly different filename, and that merging both would leave the repository with two versions of the same function. I had set up a trap for two violations. It found a third I never planted, off its own read of the wider project, and it was right. That is the moment it stopped feeling like a checkbox and started feeling like a reviewer. It would be a good idea to add rules like this to the ADR explicitly.

Let's continue. The bead went back to the pool with the failure recorded on it and the branch preserved. A fresh polecat picked it up, read the reason, and pushed a fix: it inlined the stop words, deleted the JSON file, and dropped the unrelated `titleCase` files. Then the reviewer looked again:

```
Verdict: PASS
Prior failures: 1 (fix commit 09240b3 addressed both)

Cumulative diff origin/main...origin/polecat/tt-6whx is now exactly:
  src/slugify.js        | 18 lines (new)
  tests/slugify.test.js | 24 lines (new)

Rule-by-rule: one function per file (PASS), pure with an in-file literal
and no require() (PASS), no third-party deps (PASS), one test file
covering empty, case, punctuation, separators, and stop-words (PASS).
Routed to refinery with merge_strategy=mr (main is branch-protected).
```

The whole exchange, the failure, the reason, the fix, the second look, lives on the bead. The database that coordinates the fleet is the same database that holds the argument the fleet had with itself about whether a change was allowed to land. Coordination and accountability keep turning out to be the same object.

What happens when the coder and the reviewer disagree forever? A bad implementation could bounce, get re-implemented badly, bounce again, and burn tokens in a circle with nobody watching.

So the reviewer counts. Every failure increments a number on the bead, and when it hits the cap, which I set to two, the reviewer stops bouncing and does something different: it marks the work blocked, routes it to a human, and mails the coordinator. A change gets rejected once, gets one fix, and if the second review still fails it becomes a person's problem instead of an infinite loop. In my run it never reached the cap. One bounce, then it passed. But the cap is the difference between a review loop and a way to spend money in your sleep.

## The human still owns main

A reviewer that hands clean work to the refinery is only half of trust. The other half is making sure the refinery cannot merge on its own (in the end this is up to your preference), because a fleet that can review its own work and then land it is a fleet that can talk itself into anything. And that can spiral to unknown places.

I turned on branch protection for `main`: pull requests required, at least one approving review, and `enforce_admins`. The agents run with my token. Without `enforce_admins`, "only approved reviews can merge" is a rule the fleet gets to ignore, wearing my administrative hat. With it, nobody bypasses the gate, including me, the human in charge.

When the reviewer passed the slugify change, the refinery opened a real pull request against `main`:

```
PR #6  "Add slugify string utility (tt-6whx)"
state:              OPEN
mergeable:          MERGEABLE
mergeStateStatus:   BLOCKED
reviewDecision:     REVIEW_REQUIRED
author:             mceire
files:              src/slugify.js, tests/slugify.test.js
body:               "Automated pull request published by Gastown Refinery."
```

The change is clean and it will merge fine when approved. The fleet built a correct change, walked it all the way to the edge of the main branch, and then stopped, because the last step is mine.

![Pull request #6 on GitHub, opened by the Gastown Refinery: two commits, the first attempt and the fix, and a red "Review required / Merging is blocked." The fleet cannot approve its own work.](/images/gas-city-making-it-trustworthy/pr6-blocked.png)

Open the files changed and there is nothing wrong with it: `slugify.js` with the stop-word list inlined as a literal and a single function, alongside its one test. LGTM.

![The two files inside PR #6: slugify.js and its test, exactly the diff the reviewer passed. Clean, and still waiting on a human.](/images/gas-city-making-it-trustworthy/pr6-files-changed.png)

That is where I chose to leave it. I could have added a second account to approve it and shown you a green merge, but the open, blocked pull request is the real end of the story. This is the arrangement you want: the machine does the work, and the merge stays a decision a person makes with their eyes open.

## What I'd run, and what it costs

Two more layers belong in a real setup. I did not run them, so what follows is the sketch, not the report.

First, my reviewer catches a bad change after a coder has already spent a run building it. A project-manager agent sitting in front of the coding pool would read a bead before anyone works it and reject the underspecified ones, the ones that would send a literal-minded worker into the same trap I set on purpose. It is the same pattern as the reviewer, an agent plus a formula that can say no, just moved earlier so you stop paying for work that was never going to pass. I would build this second, right after the reviewer, because a lot of bad output is really a bad bead.

Second, an opinion. A single reviewer has a single model's blind spots. Gas City lets you point different agents at different providers, so you can stand up a Claude reviewer, a Codex reviewer, and a Gemini reviewer on the same diff and only pass a change a majority approves. The provider switch is real and per-agent. The part that vote-counts across them is something you would write. You could also take advantage of GitHub's Copilot for the review. For a string library all of this would be overkill. For a change to something that pages you at night, a second and third independent read before it can land is the kind of cost I would be happy to pay.

Now, the tokens! None of this is free, and the cost multiplies. Every gate is more agents doing more passes. The bill scales with the amount of work times the number of reviewers times how many times a change bounces. My single slugify function, through one reviewer with one bounce, took around half an hour and a real slice of my daily budget.

And the fleet still needs minding, which surprised me given the whole point is to step back. The reviewer idle-stalled partway through: its session was alive, it had a bead assigned, and it simply sat there for fifteen minutes doing nothing until I sent it a nudge to go look at its queue. Same thing the refinery did in Part 2. These gates are real and they work, but "autonomous" is still doing some heavy lifting in the marketing. A person checks in.

So where would I start, on a real team. One reviewer against your decision records, branch protection with `enforce_admins` on, and the merge left to a human. That is a small amount of assembly and it changes the character of the thing entirely, from a fleet that can write to your main branch to a fleet that can only ever propose. I would add the front-door bead gate next, because it pays for itself in runs not wasted. I would add multiple vendors only for the paths where a wrong merge is expensive, and not before. I would also add local models for some coding tasks and testing. And I would not put anything I could not reconstruct afterward on a critical path, because the ground under this tooling is still moving week to week.

A fair thing to ask by now is whether any of this survives a real project, or whether it only works because my example was a library of string functions nobody will ever run. I had the same doubt.

Beads, the tracker the whole system coordinates through, is built the way this series describes. Look at its recent history on GitHub: roughly two out of every three commits carry a signature naming the model that wrote them, Claude Opus 5, Codex, Cursor, with the agents listed as co-authors, and nearly every commit tied to a bead. The work graph tracks the work that builds the work graph. And the human is still the author of record, still the one who merges. Agents write, a person owns the branch. That is the exact arrangement I spent this whole piece assembling, running on a project with twenty-six thousand stars that ships releases regularly.

This is not a factory that built itself while everyone slept. The Gas City engine underneath is still mostly hand-written. The early accounts are rough too: one of the first public trials, back when Gas Town was two weeks old, burned about ten times the tokens a person would have and left a stack of pull requests the author closed without merging one. This is frontier tooling and it bites. But the direction is not a guess. The people who build the thing coordinate through it, let the agents do the writing, and keep a human on the merge, at a scale and a pace well past my afternoon.

After three pieces, the race everyone is watching, making a single agent smarter, seems to me not so important anymore. What held my attention was the plumbing: the gates that let you trust a crowd of ordinary agents instead of babysitting one clever one. They turned out to be beads and formulas, the same parts as everything else, a reviewer that reads the architecture and can say no, and a branch the fleet cannot land on its own. You build the trust by composing the system.

If you are weighing one of these tools, that is the lens I would use. Not how autonomous it claims to be, autonomy is easy to put on a slide. Whether you can see everything it did, and whether you can insert your own gates without asking the vendor's permission. Those two are the hard parts to build, and they are the ones worth paying for. I spent a year learning to steer one agent. I spent these three pieces learning to build the loop, and then to put the fences around it. The fences are the most important part.