Unknown Confidence Is Not Medium Confidence

The primary reader is any developer designing confidence and provenance for extraction systems. The bug class is deceptively simple: converting missing measurements into a plausible numeric score.

The bug class

An extractor combines signals from native PDF objects, OCR, geometry rules, and learned models. Some sources provide calibrated probabilities, some provide uncalibrated scores, and some are factual observations with no probability at all.

A convenient fallback often appears:

const confidence = Number.isFinite(candidate.confidence)
  ? candidate.confidence
  : 0.7

The output schema now looks uniform, but the system has invented evidence. Consumers cannot distinguish a measured 0.7 from “no measurement exists.” Aggregations amplify the fiction, and review thresholds begin making decisions from it.

Why APIs encourage it

UI components prefer numbers. Sort functions dislike null. Averages become simpler if every record has a score. Static types may even encourage a required numeric field.

The semantic mistake is treating three states as two:

• measured confidence;
• measured low confidence; and
• confidence unavailable or inapplicable.

Zero does not solve the problem because zero means measured certainty of failure. A fixed midpoint is worse because it looks credible.

The fix

Make absence explicit and preserve calibration metadata:

const confidence = candidate.confidence ?? null
const record = {   confidence,   confidenceSource: confidence === null ? null : 'ocr-word-mean',   calibrationVersion: confidence === null ? null : 'ocr-cal-2026-08', }

Aggregators must exclude unavailable values and report coverage:

function summarize(records) {
  const measured = records.filter(r => Number.isFinite(r.confidence))
  return {
    mean: measured.length
      ? measured.reduce((n, r) => n + r.confidence, 0) / measured.length
      : null,
    measuredCount: measured.length,
    totalCount: records.length,
  }
}

Now mean: 0.82, measuredCount: 2, totalCount: 40 cannot masquerade as broad certainty.

Preventing the class

Schemas should use nullable confidence and require a source whenever a number exists. Tests should reject fabricated defaults. UI should render “unmeasured” distinctly from “low confidence.” Composite scores should document how missing axes are handled and must not silently renormalize away the missing evidence.

The same rule applies to model outputs. A raw sigmoid is not automatically calibrated confidence. Calibration belongs to a named model version and evaluation distribution. Synthetic validation does not prove calibration on real scans.

The general lesson is that uncertainty has structure. Preserve it. A truthful null is more useful than a precise-looking number nobody measured.