Distinguishing line art from tables in geometric PDF extraction

TLDR: PDF files do not label their drawings. A wiring diagram and a bordered table both arrive as a pile of path segments, and any geometry-based table detector will happily reconstruct a grid over a schematic. The reliable discriminator is not orientation statistics on individual segments, it is the provenance of each stroke: which kind of subpath it came from.

The problem class

Any developer building a document parser on top of a PDF renderer hits this wall. The format gives you three primitives for visual content: raster image XObjects, text runs, and paths.

Raster images are tagged. Text is tagged. Everything else, including every technical drawing, chart, logo, and decorative flourish, is just paths.

A table detector that clusters horizontal and vertical segments into a grid has no way to know it is standing inside a furnace wiring diagram. Wire runs are horizontal and vertical lines. They intersect. They form a grid by every local test you can write.

The naive approach and why it fails

The obvious first move is orientation statistics: count diagonal segments inside a candidate region and reject table candidates that contain too many.

This works until you meet two failure modes at once.

First, curve flattening. Renderers decompose beziers into polylines before you see them. A curved arrow becomes forty tiny segments, each 3 pixels long, each with dx and dy under your diagonal threshold. Your "diagonal" counter reads zero on a page that is 90 percent curves.

Second, table debris. Financial tables emit tiny axis-aligned fragments: underline end caps, rounded stripe corners, tick marks. If you loosen the filter to catch curve fragments ("anything short OR diagonal"), those fragments light up, and you paint a phantom figure over the numbers column of a balance sheet. We measured exactly this: a false figure region claimed 58 text items, all of them dollar amounts, and the table shattered into nine boxes.

The better model: stroke provenance

The fix is to stop asking "what does this segment look like" and ask "what kind of path did it come from."

When you classify subpaths before flattening them into segments, you get a small taxonomy almost for free:

// During path reconciliation, tag each emitted segment
// with the type of subpath that produced it.
canonicalSegments.push({
  x1, y1, x2, y2,
  srcType: c.type,               // RECT | FREE_PATH | POLYGON | ...
  srcSegCount: c.segsViewport.length,
});

Then the figure predicate becomes provenance plus orientation:

function isFigureStroke(s, eps = 4) {
  if (!(s.srcSegCount > 1)) return false;   // single fragments are debris
  const dx = Math.abs(s.x2 - s.x1);
  const dy = Math.abs(s.y2 - s.y1);
  const cleanH = dy <= eps && dx > eps;
  const cleanV = dx <= eps && dy > eps;
  return !cleanH && !cleanV;                 // grid lines are not evidence
}

The numbers on real documents are stark. A balance sheet page emitted exclusively rectangle-derived and dash-run segments: zero figure strokes. A wiring diagram page emitted over 2,400 figure strokes. There is no threshold to tune because the distributions do not overlap.

From strokes to regions

Individual strokes become figure regions through a coarse spatial pass:

1. Mark cells of a 16 px grid containing figure-stroke midpoints.
2. Dilate one cell and take connected components.
3. Expand each component to the extents of all member segments.
4. Gate on minimum stroke count, minimum size, figure-stroke fraction, and text coverage.

The text coverage gate matters: a bordered box full of prose has corners that flatten into curve fragments, but its area is dominated by text. A drawing's area is dominated by ink.

Accepted figures then do three jobs. Their segments leave the table-segment pool, so no grid can form over them. Their text items are claimed as labels, so captions inside the drawing stop leaking into paragraph flow. And their bounding box becomes a crop from a high-resolution canvas render, so the figure survives into the output as an image.

Tradeoffs

You give up figures drawn entirely with axis-aligned strokes. A block diagram made of pure rectangles and straight connectors will still read as geometry, and if its boxes intersect enough, as a table. In practice a downstream occupancy check catches most of these: a grid whose cells are mostly empty is not a table, whatever its lines say.

You also inherit a dependency on your path classifier. If the reconciliation stage mislabels a subpath, the figure detector inherits the error. That is the cost of provenance-based reasoning anywhere: your evidence is only as good as the chain of custody.

The alternative, a layout ML model, solves this class of problem statistically. If your architecture forbids model weights and demands determinism, stroke provenance is the strongest signal the format actually gives you.