Five document-extraction vendors publish a rule for when a machine's output can skip a human. Not one publishes evidence that the number the rule depends on means what it says.
Every claim below was read on the vendor's own domain and re-checked on 3 September 2026. Each is linked. Where a vendor's position is stronger or more careful than a summary would suggest, that is said here rather than left out.
A document-extraction system reads a payslip, a bank statement, an invoice, and returns values. Beside each value it returns a confidence — 0.82, 0.97. That number does one job in production: it decides whether a person ever sees the field.
Above the line, the value flows into a decision. Below it, a reviewer checks it. The threshold is where a company chooses to stop looking, and it is almost always set as a round number that felt about right.
Which raises a question with an answer, and the answer is measurable: of the fields a system scores 0.9, how many are actually correct? If the answer is 90%, the threshold is doing what everyone assumes. If it is 71%, then a number chosen to control risk is quietly mispricing it, and every downstream figure built on that automation rate is overstated.
Five vendors publish, in their own documentation, a rule tying a confidence score to an automation decision. Each is put to one question: does the same vendor publish data showing how often its outputs at that confidence are actually correct?
Recommends 0.95 as a starting threshold, and states plainly that the score is not a probability of correctness.
landing.aiA five-tier ladder: auto-accept at 0.95 and above; auto-accept with a 5% audit sample from 0.85; route to review below 0.85, at high priority below 0.70; reject below 0.50.
runpulse.comField-level thresholds set as validation rules, and a global threshold that auto-validates fields and whole documents meeting it.
docs.affinda.comAuto-accept and auto-reject boundaries: below the reject boundary a job becomes needs review; above the accept boundary it is marked correct. Documented as coming soon rather than shipped.
docs.super.aiLow-confidence fields routed to human-in-the-loop validation. No numeric threshold published.
klearstack.comCalibration data means published figures showing observed accuracy at stated confidence levels — a reliability table, an expected calibration error, or equivalent, for the vendor's own product. Accuracy benchmarks are a different thing and several vendors publish those: a headline accuracy figure says how often the system is right, not whether it knows when it is wrong.
Landing AI's own documentation says the confidence score “is not a probability of correctness”, and spells it out: a score of 0.80 does not mean an 80% chance the output is right. Two paragraphs later it recommends 0.95 as a starting threshold.
Both of those things are defensible, and stating the first takes more integrity than most product documentation manages. But read together they describe the whole situation exactly. The number is not a probability. The number is nonetheless where you should draw your line. How far the first fact should move the second is left to the reader, and the reader has no data with which to work it out.
Runpulse goes furthest of the five. Its post discusses expected calibration error, monitoring for calibration drift, and spot-check sampling, and it publishes a table of adjustments applied to raw scores by document type. That is a serious engineering position and better than a round number. It is still a description of method rather than a measurement of result: no observed accuracy per confidence band is shown.
An unevidenced threshold is not a neutral default. It is a live decision about how much error goes unreviewed, taken without the one measurement that would price it.
Set it too low and reviewers spend their day confirming correct extractions — a cost that is visible, complained about, and therefore usually fixed. Set it too high and wrong values flow into decisions unexamined. That cost is invisible, which is the problem: it does not appear as an error rate, it appears months later as a bad decision nobody traces back.
The asymmetry matters most in the band everyone trusts. Errors at 0.4 get caught by the process. Errors at 0.95 are, by construction, the ones nobody looks at.
Not a benchmark, and not a certification. One table, produced from data most teams already have:
That is the whole method. It is arithmetic over a table, it involves no language model at any point, and any competent engineer can run it in an afternoon. The reason to publish the output is not that it is hard. It is that a vendor who can show the line holds has evidence for a claim every competitor is currently asserting.
A calibration audit of Tesseract 5.3.4 across 200 scanned receipts — what happens when someone actually measures the number these vendors ask you to trust.