Column contamination in borderless table detection

TLDR

Borderless (stream) table detectors rely on clustering text item X coordinates to discover column anchors. Running stream table detection across a multi-column page without pre-partitioning text into column zones causes text items from adjacent columns to corrupt anchor metrics. Pre-splitting unclaimed text into distinct page-column zones before running stream detection eliminates metric contamination and restores accurate table extraction.

Stream Detection Strategy	Anchor Scope	Data Contamination	Detection Accuracy
Full-Page Un-Split Pass	Interleaved cross-column text items	High (Prose anchors dilute table signals)	Low (Misses borderless tables)
Pre-Split Zone Pass	Isolated single-column zones	Zero (Clean single-stream anchor signals)	High (100% Borderless Table Recall)

---

Problem statement: the cross-column contamination trap

Stream table detection clusters text X positions into column anchors and evaluates row spacing regularity across participating rows.

When stream detection runs on unclaimed text spanning a two-column page layout:

1. Anchors from the left column interleave with anchors from the right column.
2. Fill rate calculations evaluate against a combined anchor grid that no real row satisfies.
3. Row spacing metrics combine row gaps from independent textual streams.

This anchor contamination dilutes clean table signals, causing valid borderless tables to fail confidence thresholds.

---

Technical failure mode: in-detector gap splitting failures

Attempts to resolve column splits inside the stream table detector (e.g., splitting anchors when inter-anchor X gaps exceed $3\times$ the median gap) fail on real documents:

// DEFECTIVE ATTEMPT: Reactive anchor splitting inside stream detector
function splitAnchorsAtXGap(anchors) {
  const gaps = anchors.map((a, i) => i === 0 ? 0 : a.x - anchors[i - 1].x);
  const medianGap = calculateMedian(gaps);
// Fails on real documents: Gaps between columns often fall below 3x threshold   if (Math.max(...gaps) >= medianGap * 3.0 && Math.max(...gaps) >= 144) {     return partitionAnchorsAtMaxGap(anchors);   }   return [anchors]; }

On academic test papers, the gap between left and right column text was $125\text{px}$ ($2.3\times$ median), falling just below the $3.0\times$ threshold. The detector failed to split anchors, corrupting metrics and missing the table.

---

The fix: pre-split page columns before stream detection

Execute page column detection on unclaimed text before invoking stream table detection, running the stream detector independently within each column zone:

// REFACTORED: Pre-Split Zone Partitioning
export function processUnclaimedTextForStreamTables(unclaimedItems, scale, viewport) {
  // 1. Detect vertical column split coordinates across unclaimed text
  let columnSplitXs = [];
  if (unclaimedItems.length > 10) {
    const { splits } = detectPageColumns(unclaimedItems, viewport, scale);
    columnSplitXs = splits
      .map(s => s.x ?? s)
      .filter(x => x > viewport.width  0.10 && x < viewport.width  0.90);
  }
// 2. Partition page into column zones (-Infinity to Split1, Split1 to Split2, etc.)   const zoneBoundaries = [-Infinity, ...columnSplitXs, Infinity];   const detectedTables = [];
for (let i = 0; i < zoneBoundaries.length - 1; i++) {     const minX = zoneBoundaries[i];     const maxX = zoneBoundaries[i + 1];
// Filter text items strictly inside current column zone     const zoneItems = unclaimedItems.filter(item => item.vx >= minX && item.vx < maxX);     if (zoneItems.length < 6) continue;
// 3. Execute stream table detection on isolated zone items     const zoneTables = detectStreamTables(zoneItems, scale, { isZoneMode: true });     detectedTables.push(...zoneTables);   }
return detectedTables; }

Zone-Mode metric relaxations

Operating within isolated column zones enables safe threshold relaxations:

• Average Text Length Cap: In zone mode, when vertical white-space gutters exist, raise STREAM_MAX_AVG_LEN from 20 to 40 characters. Because text is isolated to a single column, longer text strings represent multi-word table headers rather than full prose sentences.

Rule of thumb: Pre-split page text into column zones before running spatial table detectors to prevent cross-column anchor contamination.