Human in the Loop: Designing the Review Step That Actually Catches Errors

Ask any team running a generated step in production what their reviewer has rejected recently. If the honest answer is "nothing I can remember", the review step is not a control. It is a queue with a person standing next to it, and everybody involved has been quietly reassured by a mechanism that has never done anything.

A human in the loop is a design decision, not a phrase you put in a proposal. It holds only if refusal actually happens, if the reviewer has enough information to refuse with, and if refusing costs less than waving something through. Most review steps fail that third test long before they fail the first one, and the failure is invisible because the metric everybody watches is throughput.

What follows is how we would build the step so it catches things: what to sample, which checks catch which failures, how to make saying no cheap, and why the single most effective change is almost always to make the reviewer’s job smaller rather than their attention longer. It applies to a support assistant, a content pipeline, a classifier feeding a CRM, or any automation that reaches a customer without a person in the middle.

The rubber stamp is the default outcome

Review steps decay, and they decay predictably. On day one the reviewer reads everything, because the output is unfamiliar and interesting and there is not much of it. By week three the output is familiar, the queue is longer than the morning, and reading carefully costs more than approving. Nothing changed in the model. What changed is the economics of attention.

Two forces do the damage. The first is automation bias: people accept a machine’s suggestion more readily than they would accept the identical suggestion from a colleague, and fluent writing amplifies the effect, because fluency reads as competence. The second is throughput pressure. A queue that grows faster than it clears trains the reviewer to clear it, and clearing is the thing that gets counted at the end of the week.

What a human in the loop is actually for

Be specific about the job, because "check the output" is not one. A review step exists to catch a defined set of failures before they reach somebody who will act on them. Which failures? Write the list down. It is shorter than people expect and it differs for every system; on an AI integration answering customer questions it might be three items long, while a pipeline drafting technical documentation has a different three.

It is not a quality vote

The most common design error is asking the reviewer whether the output is good. Good is unbounded. Faced with an unbounded question, a reviewer either becomes a copy editor, which is slow and aimed at the wrong target, or approves whatever reads well, which is precisely the failure mode a fluent generator produces. Replace the question entirely. Ask whether specific things are true, one at a time, and let everything else pass without comment.

This feels like a downgrade to people who care about craft. It is the opposite. A reviewer given five yes-or-no checks will catch more real defects in two minutes than a reviewer given an open brief will catch in ten, and you will be able to say afterwards which check did the catching.

Different error classes need different checks

The reason a single pass of "looks fine" catches so little is that the failures are not alike. A fabricated fact and an over-promise fail in opposite directions and are visible to completely different checks. Sort them first, then design a check for each one you care about.

Look at what the right-hand column has in common. Every entry is a yes-or-no question about one specific property. That is the only kind of question a person answers reliably at speed, and the only kind you can count afterwards. Over-promising deserves its own rule set, because it is the class most likely to create a commitment somebody has to honour later, and we have written about keeping an assistant from promising something you do not sell as a separate discipline.

Sampling: what to look at when you cannot look at everything

Full review is right at low volume and impossible at high volume, and most teams cross the line between the two without noticing they have. Once you have crossed it, sampling is not a compromise. It is the only honest option available, and it works if the sample is chosen rather than merely convenient.

Weight the sample by consequence

Random sampling is the wrong default, because it treats a price quotation and a greeting as equally worth an expert’s minute. Weight the sample towards outputs where a mistake costs the most: anything quoting a figure, anything touching a legal or safety claim, anything going to a first-time customer, anything the system itself flagged as low confidence, and anything on a topic where you already know the source content is thin. Keep a genuinely random slice alongside it, because a purely targeted sample tells you nothing about the population you stopped watching, and that population is where the next surprise lives.

When to review every item anyway

Three situations justify full review regardless of volume: the first weeks after launch, the weeks following any prompt or model change, and any output class where a single error is unacceptable rather than merely expensive. Treat the first two as temporary and write down the date they end, or full review becomes permanent by default and then decays into the rubber stamp you were trying to avoid. Some of what reaches the queue should never have been generated in the first place; a deterministic rule handles it better and needs no reviewer at all, which is the argument in where AI earns its place.

Make refusal the cheap option

Here is the change with the largest effect and the smallest cost. Look at what actually happens in your system when a reviewer rejects something. In most builds, rejection means typing an explanation into a free-text box, pushing the item into a queue nobody owns, and then following up in person if it matters. Approval means one click. You have priced refusal above acceptance and then expressed surprise that acceptance wins.

Rejection should take the same number of clicks as approval, or fewer. Give the reviewer named reasons to choose from, drawn directly from your error classes, so a rejection is one tap and a reason code rather than an essay written under time pressure. Those reason codes are also your improvement data, and they arrive free with a design you were going to build anyway.

Add one option most systems lack: send back with a correction, rather than approve or discard. A reviewer who can fix the single wrong sentence and pass the rest will do it, and the corrected pair becomes the most useful record you own. A reviewer whose only choices are accept or destroy will accept, because destroying somebody’s morning of queued work over one clause feels disproportionate, and they are right that it is.

Make the reviewer’s job smaller

When a review step misses things, the instinct is to ask the reviewer to look harder. It does not work, and it is the wrong lever anyway. The reliable improvement is to reduce what the reviewer has to hold in their head at the moment of decision. Five moves, in the order we would make them.

1. Cut the output down to one decision. Reviewing a paragraph is a judgement. Reviewing whether one claim appears in one cited passage is a check. Split multi-part output so each part is confirmed on its own.
2. Put the evidence beside the answer. The source passage, quoted, next to the sentence it supports. If the reviewer has to go and look for it, within a fortnight they will stop looking.
3. Pre-flag what the machine already knows. Low confidence, no source matched, a term from the do-not-say list, a figure appearing nowhere in the retrieved content. All cheap to detect, all useful before a person reads a word.
4. Automate the checks that are genuinely deterministic. Currency formatting, product names, forbidden phrases, required disclaimers. A person should never be asked to act as the pattern matcher.
5. Give the queue an order. Highest consequence first, oldest second. An unordered queue gets reviewed in whatever order it renders, which means the priority was set by your database.

Every one of those moves work from the person into the system, which is the correct direction of travel. A reviewer is expensive, slow, and irreplaceable at judgement. Spending them on formatting checks is a design failure rather than a staffing problem. It is also a standing cost that should be priced honestly at the start, alongside what it costs to maintain the thing you just built, because a review step is staffing, forever, not a launch task.

How to tell whether the step is working

You need evidence that the reviewer catches things, and approval rate is not evidence. Three signals are, and none of them is expensive to collect once the reason codes exist.

• Rejection reasons, by class. A distribution that roughly matches your error table is a working step. A distribution where one code is used for everything is a reviewer who found the fastest path through the form.
• Seeded errors. Introduce known-bad items into the queue deliberately and see whether they come back. This is uncomfortable to propose and it is the only direct measurement of catch rate you will ever have.
• Escaped defects. Errors found downstream by a customer or a colleague, traced back to whether that item was reviewed. Every escape is either a check you never wrote or a check that failed to fire.

A review step earns its name on the day it stops something. Until then it is a delay you have agreed to pay for and a reassurance you have agreed to believe.

Where the assistant answers from your own published pages rather than from open-ended generation, the reviewer’s job changes shape usefully: the question becomes whether the source page is right, which is a content problem with a named owner and an existing workflow. A widget such as Flidu takes that approach, answering from the website’s own content, which removes one whole error class from the reviewer’s list. It does not remove them all. A correct page can still produce a wrong answer, for reasons set out in why your chatbot gives wrong answers, and our parent company has written about the same problem from the reviewer’s side in the reviewer’s pass.

Start with three failures, not ten

If you are adding a review step to something already running, begin by writing down the three failures you are genuinely afraid of. Not ten. Three. Build the checks for those, sample against them, and accept openly that the step will miss other things, because a step designed to catch everything catches nothing at a rate anybody can measure.

Two honest limits belong in the same document. A human reviewer is not a reliable detector of subtle factual error inside fluent text: people are good at noticing a claim that contradicts something they already know, and poor at noticing one that merely has no support behind it. And review does not scale the way stakeholders assume. Doubling the volume does not double the catch rate, it halves the attention per item. If volume is growing, the answer is narrower output and better pre-flagging, not a second reviewer beside the first.

There is also a condition under which all of this reverses. If the output never leaves the team and a mistake is cheap to correct after the fact, skip the review step and spend the effort on monitoring instead. Reviewing internal drafts is a habit rather than a control, and habits consume the attention you will want later. If you are designing AI-enabled operations and want a second read on where the person belongs in the chain, tell us what the system does and what it would cost you to be wrong once.