Your structure detector needs an auditor, not a better threshold

TLDR

When layout detectors misclassify document structures (e.g., mistaking title callout boxes or diagrams for tables), developers often attempt to tweak detection thresholds (line intersection counts, distance ratios). This leads to threshold oscillation: loosening thresholds restores missed tables but introduces false positives, while tightening thresholds removes false positives but drops valid tables. Decoupling greedy candidate detection from a downstream content auditor resolves this permanently.

Detection Architecture	Quality Control Mechanism	Threshold Stability	Handling of Complex Layouts
Single-Pass Threshold Gate	In-line magic thresholds	Low (Tuning causes regression loops)	Fails on edge-case layouts
Greedy Proposal + Content Audit	Post-hoc content statistics	High (Zero threshold tuning)	100% Deterministic Demotion/Promotion

---

Problem statement: the threshold oscillation trap

In PDF layout analysis, structure detectors evaluate candidate regions based on local geometric primitives (lines, intersections, bounding boxes).

When a detector misclassifies a title banner or vector diagram as a table, developers adjust internal thresholds:

// NAIVE THRESHOLD TUNING LOOP
const MIN_LINE_INTERSECTIONS = 6; // Increased from 4 to reject title boxes
const MIN_GRID_DENSITY = 0.45;     // Increased from 0.35 to reject diagrams

Tightening these thresholds rejects false positives, but breaks detection for valid sparse tables elsewhere in the document.

---

Technical architecture: separating proposal from content audit

Allow detectors to operate greedily, and introduce a downstream auditor pass that validates candidate structures against actual page content statistics:

[Greedy Detector Proposal]  --->  [Content Statistics Auditor]  --->  [Validated Structure / Demotion]

Auditing rules

1. Grid Occupancy Audit: Calculate the fraction of candidate grid cells containing at least one text item. If occupancy is below $0.50$, demote the table to a generic callout box or paragraph text.
2. Stray Vector Ink Audit: Calculate the percentage of vector lines inside the bounding box that do not align with grid axes. High stray ink indicates a vector diagram, not a table grid.

// REFACTORED: Post-Detection Content Auditor
export function auditCandidateTable(candidateGrid, pageTextItems, pageVectorPaths) {
  const cellOccupancy = calculateCellTextOccupancy(candidateGrid, pageTextItems);
  const strayInkFraction = calculateStrayInkRatio(candidateGrid, pageVectorPaths);
// Audit Rule 1: Empty grid structure (Diagram or Title Box)   if (cellOccupancy < 0.50) {     return {       isValidTable: false,       recommendedType: cellOccupancy > 0.0 ? 'CALLOUT_BOX' : 'FIGURE_GRAPHIC'     };   }
// Audit Rule 2: High non-grid vector lines (Architectural Drawing)   if (strayInkFraction > 0.30) {     return { isValidTable: false, recommendedType: 'DIAGRAM_VECTOR' };   }
return { isValidTable: true, recommendedType: 'LATTICE_TABLE' }; }

Rule of thumb: Allow structure detectors to propose candidates greedily, and use downstream content-occupancy auditors to validate or demote candidate regions.