โ† Writing Carson Aft.
Field notesJune 2026

Where automation should stop

The interesting question was never "can AI do this task." It's which tasks a person should keep even when AI can do them.

Every automation project eventually arrives at the same argument. Someone wants to automate everything, because the demo was impressive. Someone else wants to automate nothing, because the stakes are high. Both positions are wrong in the same way: they treat the workflow as one thing. It never is.

After years of drawing this line in compliance and trust and safety, where drawing it wrong gets people hurt or companies fined, I use a rule with three tests. Take any decision inside a workflow and ask what an error costs, whether you'd catch it, and whether the decision needs a name on it.

Test one: what does an error cost?

Not the average error. The bad one. Automating a task means accepting its worst-case error at machine volume, because the machine will make its mistake a thousand times before a person makes it once. Misfiled invoice: cheap, recoverable, automate without drama. Wrongly terminated account, missed regulatory flag, harmful content cleared at scale: the cost is not the item, it's the blast radius. High blast radius doesn't forbid automation. It changes what you're allowed to automate, which leads to the second test.

Test two: would you catch it?

An error you detect quickly is a maintenance problem. An error you detect months later, or never, is a time bomb. The dangerous combination is silent and expensive: decisions where the failure produces no signal until it's structural. A lot of "AI transformation" failures are exactly this, a system that was wrong quietly, at volume, with nobody assigned to notice. If errors are detectable, build the detector first and automate behind it. If they're not detectable, make them detectable before any automation conversation happens. Sampling, reconciliation, a human reviewing a slice. Measurement isn't overhead on the automation. It's the license for it.

Automate where errors are loud. Keep people where errors are silent.

Test three: does the decision need a name?

Some decisions carry accountability that can't be delegated to a system, legally or morally. A compliance sign-off, a termination, a medical escalation, anything a regulator or a court will later ask "who decided this?" about. The honest answer can't be "the model." This is why the Equinox engine automates roughly 95% of audit work and exactly 0% of sign-offs. The system can assemble the case. A person owns the verdict. That's not a technical limitation. It's the design.

The pattern that falls out

Run a workflow through the three tests and it sorts itself. Cheap, loud, anonymous decisions: automate fully. Expensive but detectable: automate with a person reviewing the machine's work, which is where most of the throughput gains live anyway. Silent or name-bearing: the person keeps the decision, and the machine's job becomes preparing it, gathering the evidence, flagging the anomaly, drafting the case.

Notice that last category carefully, because it's the one everybody gets wrong. "A human keeps the decision" does not mean that part of the workflow stays slow. Most of the time spent on judgment calls isn't judgment. It's assembling the context to judge with. Automate the assembly and the human decision gets faster and better at the same time, without anyone's name coming off it.

The companies that get burned aren't the ones that automated too much or too little. They're the ones that drew the line by what was technically impressive instead of by what errors cost. The line isn't a technology question. It never was.