Designing human review into document AI.
Automation should make uncertainty visible. The best document AI systems know when to ask for help, and they make asking cheap. Here's how to design the review loop so automation grows on evidence instead of hope.
By Anvexis Stack Engineering8 min read
What does a confidence score actually tell you?
Less than people hope. Microsoft's documentation for its Document Intelligence service describes field confidence as an estimated probability that the prediction is correct, and gives the example that 0.95 means the prediction is likely correct 19 out of 20 timesSource [1]. It also recommends using confidence to decide whether to accept a prediction automatically or flag it for human review when accuracy is criticalSource [1].
Do the arithmetic for your own inbox. At a thousand invoices a month with ten fields each, even a model that is right 95% of the time on every field produces hundreds of wrong values. The question isn't whether errors happen. It's whether your workflow catches them before they reach the ledger.
Read the scores carefully, too. Microsoft notes that the confidence of the text-reading step and the confidence of the key/value extraction are separate, and you should check bothSource [1]. A perfectly read number placed in the wrong field is still wrong.
Why full automation is the wrong first goal
It's tempting to judge a document AI project by one question: how many documents can it process with nobody looking? That framing pushes teams to automate too early, before they understand where the model fails, and the first costly mistake destroys trust in the whole system.
A better first goal is a faster, more consistent review. Every document gets extracted, validated, and presented to a reviewer with the fields filled in and the doubtful ones highlighted. Reviewers confirm instead of typing, and the system learns where it can be trusted. Automation then grows in measured steps, document type by document type, as the numbers justify it.
The review loop
Define fields and rules
What must be extracted, and which business rules each value must pass: totals that add up, a known supplier, a purchase order that matches.
Score confidence
Combine model confidence with rule results into one routing decision, per field and per document.
Queue exceptions
Uncertain or rule-breaking documents go to reviewers with the problem fields highlighted and the reason shown.
Capture corrections
Every fix is stored as labelled data for evaluation and improvement, with the original value kept.
Confidence is not a single number
Models return confidence per field, and it's tempting to set one threshold for everything. In practice, reliability varies by field. A model may be excellent at invoice dates and much weaker at splitting line items or reading tax amounts on unusual layouts.
Track accuracy per field and per document type, and combine model confidence with business rules. A field the model is unsure about but that matches the purchase order exactly can pass. A field the model is very sure about but that breaks a rule, such as line items that don't add up to the total, still needs a person. Microsoft's own guidance points the same way for sensitive cases like financial records: target accuracy close to 100% and add a human review stage for critical workflowsSource [1].
Borrow the trigger patterns the big platforms use
Amazon's human-review integration for its Textract service is a useful reference design, even if you never use it. Its review triggers include confidence for specific form fields falling within a chosen range, required fields being missing, any field scoring within a range, and a random sample of documents sent for review regardless of scoresSource [2].
That last one matters. Random sampling is how you find out whether your confident, auto-approved documents are actually right. One more practical note from the same page: AWS says the integration is no longer open to new customersSource [2]. Vendor review features come and go, so keep your routing rules in your own system, where they survive a change of model or provider.
A routing policy example
| Condition | Route |
|---|---|
| All rules pass, high confidence on every field | Auto-post as a draft |
| Any business rule fails | Review queue, failing rule shown |
| Low confidence on totals or tax | Review queue, high priority |
| Required field missing | Review queue, field highlighted |
| Unknown supplier or changed bank details | Supplier verification queue |
| Random sample of auto-posted documents | Spot-check queue |
Route by risk, not just by confidence
Not every document carries the same risk. A low-value delivery note with a slightly uncertain date isn't worth a reviewer's time; a large invoice with new bank details always is. Combine confidence with business risk factors such as amount, supplier history, and which fields changed, so review effort goes where mistakes would cost the most.
Make the routing rules visible and adjustable by the finance or operations team, not buried in code. They understand the risks better than any model, and they'll want to tune the rules as they gain experience.
Design the reviewer's screen first
Reviewers spend their day on one screen, so its design decides whether the system saves time. Show the original document beside the extracted fields, highlight where each value was found, and put uncertain or rule-breaking fields first. Keyboard shortcuts for approving and moving to the next document matter more than most model upgrades.
Every correction a reviewer makes is valuable data. Capture the original value, the corrected value, and the reason when possible, and feed them back into the evaluation set and the rules. You can try the reviewer flow yourself in our document AI sandbox.
Measure the workflow, not just the model
Track the review rate, the time per review, and the correction rate per field. A model that's slightly less accurate but routes uncertainty well can beat a stronger model that's confidently wrong, because the expensive errors are the ones nobody looked at.
These numbers also make the business case honest. Faster review is a real saving from week one, long before any document is fully automated.
What to track from day one
| Measure | What it tells you |
|---|---|
| Field accuracy by document type | Where the model is reliable |
| Review time per document | Whether reviewers are getting faster |
| Correction rate by field | Where rules or prompts need work |
| Auto-approval rate | How much work has been safely removed |
| Spot-check failures | Whether automation is drifting |
Grow automation with evidence
Once a document type has a long run of reviewed results, you can see exactly how often each field was right. If a supplier's invoices have been extracted correctly for months, they may be safe to approve automatically when every rule passes. A new supplier, a new layout, or a failed rule sends the document back to review.
Keep sampling auto-approved documents for spot checks. Accuracy drifts when suppliers change templates or a model is updated, and regular sampling catches drift before it reaches the ledger.
OCR, layout models, or language models?
There are three broad ways to read a document. Classic OCR turns pixels into text and leaves you to find the fields. Layout-aware extraction models, like the cloud services mentioned above, return named fields with confidence scores. General language models can read almost anything and explain what they found, but they need strict output schemas and validation to be safe in a workflow.
Many good pipelines combine them: a layout model for well-known document types, a language model for the long tail, and the same rules and review queue behind both. The review loop is the stable part. Models will keep improving and changing; the routing, rules, and audit trail are what make any of them trustworthy.
Plan for the documents nobody expected
Inboxes contain surprises: a credit note instead of an invoice, a statement in a different language, a photo of a receipt taken at an angle, or an email with the invoice pasted into the body. Classify documents before extracting, so each one goes through the right schema, and send anything unrecognized straight to a person with a clear label instead of forcing it through the wrong path.
Over time, the unrecognized pile shows which document types are worth supporting next, based on real volumes rather than guesses. Treat document content as data, never as instructions; text inside a supplier's PDF should never be able to change what your pipeline does, a point we cover in building production AI agents.
Keep an audit trail
For every document, keep the original file, the extracted values, the rules that passed or failed, any reviewer changes, and who approved it and when. Auditors and accountants will ask how a figure entered the system, and the trail answers in seconds.
The same trail is your best tool for improving the system. Patterns in corrections show which suppliers, fields, or rules need attention next.
Start with one document type
It's tempting to launch with every document the business receives. Starting with one high-volume type, usually supplier invoices, gives the team a clear success measure, a focused review queue, and a chance to learn how the whole loop behaves before adding complexity.
Bring finance into the design from the start. The people who approve payments know which mistakes are costly and which are harmless, and their trust is what ultimately lets automation grow. Our document intelligence projects begin with exactly this workshop.
Frequently asked questions
What confidence threshold should we use for auto-approval?
Does human review defeat the point of document AI?
How do we catch errors in documents that were auto-approved?
Which documents should we automate first?
Sources
- [1]Interpret and improve model accuracy and confidence scores (Document Intelligence) (opens in a new tab)Microsoft Learn, 8 April 2026
- [2]Use Amazon Augmented AI with Amazon Textract (opens in a new tab)Amazon Web Services documentation
Written by Anvexis Stack
We build Odoo, AI, web, and cloud systems for growing companies, and we write about the patterns we use. This is engineering guidance, not a product guarantee: check vendor details for your version before you decide. Last reviewed .
Keep exploring
Document intelligence
Extract data from invoices, purchase orders, and contracts with AI, validation rules, confidence scores, and human review queues.
LabsAI document sandbox
Explore an AI document extraction review: synthetic invoice and purchase order fields, confidence signals, and how uncertain results go to review.
AIAI security & governance
Deploy AI safely: private model hosting, data minimization, access controls, prompt-injection defenses, audit logs, and usage policies.
Next step
Got a tangled system? Let's untangle it together.
Thirty minutes with an engineer, no slide deck required. You'll leave with a clearer picture, whether or not we work together.
