The fill rate gate that let prose through

TLDR

Stream table detection uses fill-rate metrics (occupied grid cells divided by total grid cells) to reject false positives. However, when PDF encoders store text as individual word spans, a single dominant left-margin alignment combined with coincidental word positions creates multiple "column anchors." This inflates the cell count, allowing prose paragraphs to pass the fill-rate gate. Adding a Column Anchor Quality Gate, requiring column anchors to appear across a minimum percentage of rows, eliminates false positives.

Cell Detection Metric	Target Evaluation	Vulnerability	Gate Optimization
Global Fill Rate	taggedCells / (rows * cols)	Skewed by single dominant margin	Passes prose paragraphs with word spans
Anchor Quality Gate	Row count presence per anchor	Requires broad vertical presence	Rejects false word-span column clusters

---

Problem statement: the prose fill-rate defect

When processing PDF documents generated with granular word-level spans (where every word has an independent X coordinate), stream table detectors discover coincidental X alignments across lines.

A 16-line prose paragraph with a dominant left margin at $X=64\text{px}$ and 8 coincidental word-position alignments evaluated as follows:

Candidate Stream Table:
Total Bands (Rows): 16
Discovered Column Anchors: 9
Anchor Band Counts: [14, 2, 2, 2, 2, 2, 2, 2, 2]
Calculated Fill Rate: 0.304 (Exceeds 0.30 threshold!)

Because 14 of 16 rows matched the left-margin anchor, the total item count pushed the global fill rate above 0.30, classifying standard body paragraphs as borderless stream tables.

---

Technical failure mode: distribution skew in fill-rate math

Global fill-rate calculations measure overall grid density without evaluating column distribution:

// DEFECTIVE IMPLEMENTATION: Global fill rate evaluation only
const totalPossibleCells = rowBands.length * columnAnchors.length;
const fillRate = occupiedCells.length / totalPossibleCells;
if (fillRate >= 0.30) {   // Passes prose paragraphs with single dominant left margins!   return promoteToStreamTable(rowBands, columnAnchors); }

A single column anchor appearing in 14 rows, combined with 8 noise anchors appearing in only 2 rows, satisfies the global density ratio despite lacking true multi-column structure.

---

The fix: column anchor quality gating

Filter discovered column anchors by requiring them to span across a minimum percentage of document rows before evaluating table validity:

// REFACTORED: Column Anchor Quality Gate
function validateStreamTableAnchors(rowBands, columnAnchors) {
  // 1. Calculate minimum required row presence threshold
  const minRowPresence = rowBands.length <= 5 
    ? rowBands.length 
    : Math.max(3, Math.floor(rowBands.length * 0.25));
// 2. Count distinct row presence per column anchor   const qualifiedAnchors = columnAnchors.filter(anchor => {     const distinctRowIndices = new Set(anchor.items.map(item => item.bandIndex));     return distinctRowIndices.size >= minRowPresence;   });
// 3. Require at least half of the discovered anchors to qualify   const requiredQualifiedCount = Math.ceil(columnAnchors.length / 2);      if (qualifiedAnchors.length < requiredQualifiedCount) {     return null; // Reject false stream table candidate   }
return qualifiedAnchors; }

For the failing prose paragraph:

• Total bands: 16

minRowPresence = Math.max(3, Math.floor(16 0.25)) = 4

• Qualified anchors with presence $\ge 4$: 1 (the left margin)
• Required qualified anchors: $\lceil 9 / 2 \rceil = 5$
• Result: Rejected.

Rule of thumb: Combine global grid fill-rate thresholds with column anchor quality gates to prevent single dominant margins from misclassifying prose text as stream tables.