The Name of the Frame

Why is test-driven development effective? Here’s one belief I have: It forces us to think about correctness before we think about the path to correctness. It sets the right frame.

Attention can be steered without our noticing. Psychologists call this ‘priming’: the pattern you’ve just been thinking about shapes what you think next. Try this: say these words out loud: mop, top, crop, flop.

Now, what do you do at a green light?

If you did not say ‘go’, you’ve been primed. TDD guards against this by writing tests first. With tests last, those tests are primed by what you built, not what you meant. Agents have the same challenge, and having the agent write its tests first only goes so far; they’re still written from the agent’s own reading of the task. That’s why validation of acceptance has to come from outside the implementing agent. If we do that, we can make priming work for us instead of against us.

Regulators and Effectors

In my last article, I spoke about splitting the practice of review into three disciplines:

1. Diagnosis
2. Understanding
3. Repair

In cybernetics terms, the thing that takes action on a system is called an effector; that’s our implementer agent, which is the ‘Repair’ discipline broadened. To control a system it is paired with a cybernetic regulator: it compares the result (code produced by the effector) against a policy (coding standards) and set point (specification). This is our validator agent, taking on the ‘Diagnosis’.

For software agents, the regulator and effector must share a common understanding of the goal (i.e. acceptance criteria in the spec). However they should not share their methods. The regulator doesn’t know how the work was carried out, and the effector doesn’t know which tests were used for the diagnosis. We’re specializing. Specialization focuses time and attention on doing one thing very well and scaling it. Notice the word ‘attention’; attention is cognitive energy directed at something specific in an unbroken time frame.

Second-order regulation

Last time, I argued that a review shouldn’t generate work that nobody budgeted for. I advocated for a cap to the number of rounds the validator and implementer can go back and forth in a loop. Here I offer one way to do that while simultaneously preserving human judgement on top. It takes the form of a third agent role, the adjudicator, for governing that loop. It carries the ‘Understanding’ discipline. The adjudicator has access to the spec and the feedback passed back and forth. Its job is to escalate to you, either at a policy-based deterministic threshold, or if it detects disagreement of intent. This is how you stay in the loop, and stay in control.

Model of correctly-primed attention

0. prerequisite: a common definition of the goal (humans own the specs)
1. implementer does work to fulfill the goal
2. validator checks the work
3. validator either
   a. accepts the work (done), or
   b. sends scoped* feedback to the adjudicator
4. adjudicator either
   a. passes the feedback to the implementer, who repairs (goto 1), or
   b. stops the loop and escalates to you when the round cap is reached
      or it spots a disagreement over what the spec means (goto 0)
      
*the feedback is about what went wrong, not about how 'wrong' was determined

Acceptance: The Spec

As always, this starts with a specification, a well-defined description of the goal and constraints of the desired change. There are many ways to do this, and most share common traits. I personally like this shape:

Feature: As a shopper, I want to see how close my cart is to free shipping,
so that I can decide whether to add another item before checking out.

Definitions --
Free-shipping threshold: $50.00

Acceptance Criteria --
1.
GIVEN a cart with a subtotal of $42.00
WHEN the shopper views the cart
THEN the cart shows "Add $8.00 more for free shipping"
AND shipping is charged at the standard rate

2.
GIVEN a cart with a subtotal of $50.00 or more
WHEN the shopper views the cart
THEN the cart shows "You've got free shipping"
AND shipping is $0.00

Imagine this spec adds a feature to an existing e-commerce application. That application already supports discounts. Do these criteria say anything about discounts? How will it implement the free shipping check if a discount brings the subtotal below the threshold? The implementer will make a choice, and its own tests will pass either way, because they’re written from its own choice. Spell it out, and the validator can catch the wrong one. For instance:

subtotal: merchandise total after discounts

3.
GIVEN a cart with $55.00 of merchandise
WHEN the shopper applies a discount code that brings the subtotal to $45.00
THEN free shipping is removed
AND the cart shows "Add $5.00 more for free shipping"

Attention

With our three agent personas and the goal defined, next we need to ensure each agent has an optimized frame of reference. What does each need to do its job?

The implementer needs to know the spec, the codebase it’s changing and, after the first pass, the feedback from the validator. It does not need to know there is a loop. It knows what it will be judged against, not how. I believe you probably want this agent writing mostly unit tests. It hands off the change set for evaluation.

The validator needs to know the spec, the product, and the change set. It treats tests in the change set as evidence, not proof of acceptance. It needs its own way to check for validity. I suspect the strongest approach is a set of acceptance tests handed to it; this is an opportunity to introduce another human on the loop. After checking, the validator hands back feedback on which criteria are unmet.

The adjudicator needs to know the context of the hand-offs between the other two agents. It needs to know about the spec, so that it can recognize disagreements in interpretation. You must be the one to rule on that, probably meaning the spec has to be updated. It needs a policy that defines the limits for the workflow, and the method of escalation/notification.

Key moves

1. Write acceptance before the work starts, from a different source. Human-authored acceptance criteria are critical for capturing intent and staying accountable. If you leave them out, the agent will fill in the gaps for you.
2. Decompose. Smaller slices of work are cheaper to repair.
3. Three agent personas.
   1. Implementer - an effector of change
   2. Validator - a regulator of change
   3. Adjudicator - a regulator of regulators
4. Protect the frame. Give agents precisely what they need to do the job, no more. If you leave it in, they will use it.
5. Treat agent-written tests as evidence, not acceptance. Unit tests are still valuable for an implementing agent, as long as they remain fast and cheap. It’s a form of ‘showing the work’. Tag them as such, and review them like production code.
6. Budget fix capacity. Don’t let an overpowered regulator “Well, actually…” you for 20 review rounds. Use that as a signal that something is wrong.

Conclusion

These roles aren’t new. We’ve had developers, QA and tech leads for a long time, and that’s what these are: specializations, each with a context built for its job. What’s new is that we have to build that context deliberately, because a language model will use whatever we leave in it.

Remember the green light: the answer depended on what you’d just been saying. TDD set the frame by putting the test first. With agents, you set it by making sure each one’s context starts from the goal, not from the work.

The adjudicator is the piece that matters most. It’s the regulator of regulators, and it’s how you keep your seat at the table. If you can’t make the process accountable to you, you don’t truly control it. Your job is to be the prime regulator: own the goal, rule on what it means, and decide when the loop has run long enough. That’s where the engineer’s role is headed.