When Better Detection Metrics Produce Worse Documents

The primary reader is an engineer training a detector inside a larger extraction system. The lesson is how a locally correct metric can select the wrong model for the product.

The assumption that seemed reasonable

The team trained a compact page-layout detector. The acceptance target was clear: beat a larger incumbent on degraded pages while staying small enough for browser delivery. Mean average precision was familiar, reproducible, and cheap to calculate, so validation mAP selected the best checkpoint.

The detector succeeded at that target. It found labeled regions on degraded pages far better than the incumbent. The model was small and fast. By ordinary detector criteria, the project had won.

When it failed

The downstream extractor told a different story. Text coverage was lower. More text was left orphaned. Table cells attached less reliably, and reconstructed tables contained more weak structures. The detector produced boxes the benchmark liked, but the assembler could not use them as effectively.

This was not a measurement bug. It exposed a mismatch between the training objective and the deliverable. Intersection-over-union treats an undersized paragraph box and an oversized paragraph box somewhat symmetrically. The assembler does not. An oversized box may include harmless whitespace; an undersized box can drop two lines and create orphaned text.

What was actually wrong

Detection was being treated as the product rather than one source of structural evidence. The true output depended on containment, attachment, reading order, and table topology. None of those appeared in checkpoint selection.

The architecture also represented regions as a mostly flat list. A table inside a form looked much like a table beside a form. A caption near a picture had no explicit edge to that picture. Geometry heuristics downstream had to reconstruct relationships the model never expressed.

The core failure was therefore structural: a local box metric had been promoted into a global product metric.

What gets replaced

Validation mAP remains useful, but it loses sole authority. Checkpoint selection must include representative downstream measures:

• source-text coverage;
• orphan rate;
• fragmentation;
• reading-order quality;
• table fill and span correctness;
• weak-table count; and
• form label/value accuracy.

Training also needs asymmetric coverage pressure. Missing source content is generally more expensive than including modest whitespace. That bias can be introduced in box loss and assignment without adding model parameters.

Flat detections are supplemented with relationships. The system explicitly represents caption-to-picture, cell-to-table, field-to-form, label-to-field, and reading-order edges. Deterministic constraints then reject cycles, illegal parent types, and inconsistent alignment.

What replaces it

The new evaluation ladder has three levels:

1. Detector gates verify that learning is stable and boxes remain competitive.
2. Structural gates measure coverage, relations, tables, forms, and reading order.
3. End-to-end gates measure the exported artifacts users actually consume.

No single composite should hide a collapse in one critical axis. A release can require a minimum on each core measure, then use a composite only to rank checkpoints that already pass.

Visual review remains mandatory. Some failures are obvious in an overlay and invisible in aggregate statistics: permuted class labels, shifted coordinate transforms, relationship edges pointing across columns, or box boundaries consistently cutting through the last line of text.

Checkpoint evaluation must also match production distributions. A degraded validation set answers whether augmentation robustness improved. A clean set answers whether that robustness displaced ordinary document behavior. A real downstream corpus answers whether the assembled result improved. Mixing them into one unlabeled average would recreate the same mistake at a higher level.

The practical cadence is inexpensive: calculate detector metrics every epoch, run a small representative downstream subset every few epochs, and reserve the full corpus for release candidates. Store every component score beside the checkpoint hash. This makes it possible to choose a model for a declared workload and prevents a later benchmark run from becoming detached from the exact weights it evaluated.

Finally, the team should define adoption gates before seeing results. If the candidate must not regress clean-page extraction, state that threshold first. Moving the threshold after training turns evaluation into model advocacy instead of evidence.

The generalizable lesson

Machine-learning metrics are contracts with the optimizer. If the metric does not encode the cost experienced downstream, the optimizer is free to win the metric by making the product worse.

Before increasing capacity, adding experts, or training longer, trace every model output to its consumer. Determine which errors are cheap, which are asymmetric, and which require relationships rather than better classification. Then make checkpoint selection reflect those facts.

A detector is successful only when the system using its detections improves. Everything else is an intermediate score.