In a legal AI agent and in a photo-based ordering system, the rule was the same: everything the machine produces is a proposal until a person approves it. The work is deciding where that approval sits, what gets locked, and what shows up before the click.
Anderson Silveira · October 2, 2026
A law firm handles debt enforcement cases from both sides: it collects for creditors and defends debtors who are being sued. Creditor and debtor cases trigger on the same terms, “bank enforcement,” “credit note.” If the AI inferred the side from the text and got it wrong, it would produce a filing defending the very party the client wants to collect from.
That’s the kind of error that decides the design of an AI product. So in the legal agent I built for this firm, the side of the case is never inferred. It’s a human choice, every time.
The general rule came out of that and repeated itself in a very different system, a distributor’s photo-based ordering: the AI proposes, a human confirms. This article is about how that sentence becomes a screen, not a footer disclaimer.
Everything is a draft until someone approves it
The lawyers were already using AI before the product, each in their own chat. What was missing wasn’t reading. It was control: nothing stayed with the case, nobody knew what an analysis would cost before asking for it, and a sentence without the page it came from forced them to reread the entire case file.
The first decision was about vocabulary. Everything the machine writes is a draft. The interface says so on every analysis, every chat response, every document.
Approval is an act of the lawyer, with name and time on record. Next to every analysis, a panel called the “human gate” shows who approved it and when, how many references still need to be checked against the original, and how many findings have no anchor. Until someone approves, the panel reads “no reviewer yet.” When someone does, a green bar records the handoff of responsibility: the analysis stops being a draft and belongs to whoever signed off on it.
- 01The machine proposes
An analysis, an order line, a lesson. Always labeled as a proposal.
- 02The screen shows the source
The passage and page from the case file; the text on paper next to what was understood.
- 03The person checks
Line by line, finding by finding. Whatever wasn’t recognized blocks the flow.
- 04The person takes ownership
Approve or confirm, with name and time on record.
- 05Only then does it count
The analysis becomes an opinion; the order enters the store’s queue.
Where to put the review
Saying “a human reviews it” isn’t enough. The question is at what point on the screen, and in what order.
In the legal agent, the approve button sits at the end of the page, after all the content, and the text next to it explains what the click means. The full reading of the filing, in document typeface, only prints and downloads. It doesn’t approve. The order of the page is the order of responsibility: read, check, take ownership.
On the phone, the lawyer checks the list, reads the triage, and downloads the document. But the screen itself says that approval doesn’t happen on the phone, and neither does drafting the filing. The draft is meant to be reviewed section by section, on a screen where that fits.
In the photo order, the review sits between the reading and the order. The photo of the note stays at the top, with zoom, and each line shows what was written on paper next to what the system understood. A doubtful line appears as a suggestion: “We’re not sure. Is this the one?”, with an editable quantity and the options to accept, swap, or ignore. Once confirmed, the order reaches the store’s dashboard like any other, with the original photo attached so the picker can check it.
What to lock
Not everything is a matter of review. Some things the machine isn’t allowed to do, and the system refuses.
| What the machine does on its own | What it only proposes | What only the person does |
|---|---|---|
| Reads the case file in chunks and points to the passage and page of every finding | The triage conclusion, the draft filing, the chat answer | Chooses the side of the case: creditor or debtor |
| Matches each line of the photo against the catalog | The product and quantity of each recognized line | Decides the line the system didn’t recognize |
| Compares generated text with approved text and suggests lessons | An instruction for the case or a lesson for the firm | Approves the analysis; promotes the lesson, if a partner |
An unrecognized line never becomes a guess. In the photo order, it’s flagged in red, asks the buyer to pick the product or ignore the line, and the continue button only unlocks when no line is left undecided.
An answer without a source is a bug. In the legal agent, every conclusion cites the passage and the page. An answer without an anchor is treated as a serious bug, not a matter of style. An unreadable scanned page is a declared limitation, never silence.
Learning goes through the gate too. When a conversation with the agent reveals something worth keeping, it proposes an instruction for the case or a lesson for the whole firm. Nothing is saved on its own. The lawyer accepts with one click, and only a partner promotes a lesson to active.
Cost visible before running
Control also means knowing what something costs before asking for it.
In the legal agent, before any analysis, the run panel shows the path (documents, analysis, draft, review), what the run will read, which document is image-only, and how much it should cost, against how much is still left in the case’s budget. It’s a worst-case estimate, calculated from the largest run the case has had so far, and the screen says so. Only then does the run button appear. Every case has a budget cap.
Control runs both ways
“Human in control” is usually read as the person correcting the machine. The opposite happens too.
The creditor’s full workflow was validated end to end on a real case: triage, legal opinion, audit, and an approved draft on the firm’s letterhead. In one of the real runs, the filing plan flagged as a critical alert a deadline error that was in the human instructions themselves.
That was only possible because the machine doesn’t execute instructions silently. It proposes, points to what it saw, and waits. When the proposal and the instructions disagree, the disagreement surfaces for someone to look at.
What this costs
Reviewing line by line is slower than accepting everything. Approving only on the computer is less convenient. Choosing the side by hand is a click the AI could have saved.
In both projects, that price was accepted because the error on the other side was bigger: a filing defending the wrong party, a 200-liter drum instead of a 20-liter bucket. Where a mistake is cheap and easy to undo, the math may come out differently. The point is to decide, with the cost of a mistake on the table, where the person needs to be.
What I take away
“Human in control” isn’t a disclaimer in the footer. It’s a design decision that shows up on every screen: the draft label, the gate that counts anchors, the approve button at the end of the page, the proposal waiting for a click, the lesson waiting for a partner, the cost that appears before the button, the phone that lets you read but not sign, the red line that blocks the order.
None of these pieces is hard to build. The hard part is deciding they exist before the first analysis comes out looking polished and convincing on screen. After that, the temptation to trust it is too strong.
The legal agent case and the ordering system case show the screens. In Reading documents in bulk with AI, I get into what the machine reads and where it goes wrong.
Checklist for putting people in control
- Is every machine output labeled as a proposal, on the screen itself?
- Is approval an act with a name and a time, not a convenience button?
- Is the source of every claim right next to it: passage and page, or the original next to the reading?
- Does whatever the machine didn’t recognize block the flow, instead of becoming a guess?
- Are the highest-cost decisions (the side of the case, the packaging) always human?
- Does the cost of a run appear before the button, against a cap?
If the answer to any of these is no, the person is in the flow, but not in control.