Automation that drafts; a person who decides.
Human-in-the-loop AI. Human-in-the-loop AI is an automation design in which a person reviews, corrects or approves an AI system’s output before that output takes effect. In customer support, human-in-the-loop AI usually means the AI drafts a reply and a team member approves or edits the draft before the customer sees anything.
The AI produces a candidate action — a drafted reply, a proposed refund, a suggested category — and instead of executing, queues it for a person. The reviewer approves, edits or rejects; only then does the action happen. Each decision doubles as feedback: edits show the system what it got wrong, approvals confirm what it got right, and over time the corrections become the evidence for automating more.
The real design question is not whether to have a human in the loop but where the gate sits. Three positions exist: before every action (approval), after the fact on a sample (quality review), or only when the AI is uncertain (exception handling). Mature teams move the gate per intent, not per tool — password-reset replies may lose their gate within weeks, while anything touching money or account deletion keeps a gate indefinitely. This is the model behind inrelay’s approve before send workflow: the AI drafts, a person releases.
Language-model errors are fluent — a wrong answer reads exactly as confident as a right one, which defeats the skim-reading a busy human naturally does. A review gate converts that risk into edits: mistakes become corrections a customer never sees, rather than apologies after the fact. The gate also solves the trust problem in both directions — the team learns what the AI can be trusted with by watching it, instead of guessing from a demo.
The failure mode has a name worth knowing: approval fatigue. When a reviewer approves fifty drafts in a row without a single edit, approval degrades into rubber-stamping — and a gate nobody engages with is worse than no gate, because the mistake still ships, now stamped “a human approved this.” A loop where the human no longer looks is an accountability laundry.
Start with everything gated and measure edit rate per intent. Where the edit rate stays near zero for weeks, loosen the gate for that intent — move it from approval to sampled review — and say so explicitly, so the change is a decision rather than drift. Keep irreversible or high-stakes actions gated regardless of edit rate: the cost of a rare error, not its frequency, is what earns a permanent gate.
Guard the reviewer’s attention like the scarce resource it is. Keep review queues short, show the customer’s message beside the draft so review means reading, and track edits as the system’s report card — a rising edit rate is your earliest warning that something upstream changed.
Human-in-the-loop is a control pattern applied to increasingly autonomous systems — these entries cover what sits on either side of the gate.
Give your customers faster answers and your team their evenings back.
Desktop and mobile. Free for solo.