How to Stop PDF Parsers from Hallucinating Tables out of Thin Air

TLDR

Standard PDF table extractors misinterpret decorative rules and underlines as grid line segments, hallucinating phantom 1x1 tables across body copy. The contextClassifier prevents phantom table generation by evaluating proximity between text baselines and horizontal vector paths ($0\text{ to }5\text{px}$ below baseline), removing detected underlines prior to lattice table reconstruction, and scoping unclaimed text into non-overlapping spatial region buckets.

Parser Architecture	Underline & Rule Handling	Table Extraction Strategy	Phantom Table Risk
Sequential Un-Scoped Extractor	Treats all horizontal vector lines as borders	Global lattice reconstruction pass	High (Creates phantom 1x1 tables)
Context-Aware Region Classifier	Proximity pre-pass removes underlines ($0\text{ to }5\text{px}$)	Scoped bounding box text mapping	Zero (Deterministic region isolation)

---

Problem statement: the phantom table phenomenon

Naive PDF table extractors evaluate vector line segments independently of text content.

When a document contains decorative rules or underlined headings:

1. False Lattice Generation: The lattice engine sees horizontal vector segments and attempts to construct grid intersections.
2. Text Misclassification: Paragraph headings with underlines get misclassified as 1x1 lattice tables.
3. Data Loss: Text slightly offset from cell centers fails strict bounding-box checks and drops out of the extracted output entirely.

---

Technical architecture: proximity-based underline filtering

Filter decorative vector underlines before running table reconstruction passes:

// PROXIMITY PASS: Tag and filter underlines (0-5px below text baseline)
export function filterUnderlineVectorSegments(hSegments, textMetaItems) {
  const underlineIds = new Set();
hSegments.forEach(h => {     const hY = (h.y1 + h.y2) / 2;
for (const item of textMetaItems) {       const yDistance = hY - item.vy;
// Underline Condition: Line sits 0 to 5px below text baseline with overlapping X span       if (yDistance >= -1 && yDistance <= 5 && isXSpanOverlapping(item, h)) {         underlineIds.add(h.id);         break;       }     }   });
// Return vector segments excluding decorative underlines   return hSegments.filter(h => !underlineIds.has(h.id)); }

---

Non-Overlapping region scoping & proximity cell snapping

1. Spatial region scoping

Claimed text items are tagged in a central assignedTextIndices bitset, preventing items inside tables from leaking into surrounding paragraph blocks:

const tableTextIndices = [];
textMetaItems.forEach(item => {
  if (isPointInsideBBox(item.vx, item.vy, tableBBox)) {
    tableTextIndices.push(item.idx);
    assignedTextIndices.add(item.idx); // Mark consumed
  }
});

2. Nearest-Neighbor cell proximity snapping

To handle rendering coordinate jitter where text sits $0.1\text{px}$ outside cell borders, use Euclidean proximity snapping ($15\text{px}$ search radius) to assign text to the nearest cell center rather than requiring absolute containment:

export function snapTextToNearestTableCell(textItem, cellGrid, snapRadiusPx = 15) {
  let minDistance = Infinity;
  let targetCell = null;
cellGrid.forEach(cell => {     const cellCenterX = cell.x + cell.w / 2;     const cellCenterY = cell.y + cell.h / 2;     const distance = Math.hypot(textItem.vx - cellCenterX, textItem.vy - cellCenterY);
if (distance < minDistance && distance <= snapRadiusPx) {       minDistance = distance;       targetCell = cell;     }   });
return targetCell || fallbackContainmentSearch(textItem, cellGrid); }

Rule of thumb: Remove decorative underlines using baseline proximity math before running table reconstruction, and use nearest-neighbor snapping to place text in cells.