Pdf Processor

Direct Manipulation on a Spatial Canvas: How to Build a Correction Layer Over an Automatic Pipeline

2026-06-01

TLDR

Automatic pipelines get most cases right and some cases wrong. Adding more tunable parameters to fix the wrong cases contaminates the right cases. The better pattern: run the pipeline automatically, let the user correct the specific regions it got wrong by pointing at them directly. This post covers the implementation: coordinate spaces, handle geometry, override persistence, and manual bypass for the detector that fails most often.

Repo: tools/pdf-processor

The Problem Class

Any developer building a document processing pipeline will encounter this: the algorithm produces bounding boxes or regions that are wrong for some documents. The instinct is to add a tunable parameter. Adjust a threshold, re-run, check if it improved.

The problem with tunable parameters for regional errors: they are global. A column gap threshold that fixes a narrow-gutter document will break a standard two-column layout. There is no global value that is correct for all documents, because the error is structural to each document, not systematic across all documents.

The correct model for regional errors is direct manipulation. The user clicks the region that is wrong, corrects it in place, and re-runs. The correction is local. Nothing else changes.

The Two-Mode Canvas

Two interaction modes, mutually exclusive via toolbar toggle:

Select mode: interact with existing regions. Click to select, drag handles to resize, drag center to translate, keyboard shortcut to reclassify, delete key to suppress.

Column split mode: draw vertical lines that override the automatic column detector entirely for this document.

Both modes share one constraint: all interactions happen in canvas pixel space, but corrections must be stored in the pipeline's coordinate space so they survive re-extraction at different canvas sizes.

Coordinate Spaces

The canvas renders at CSS display size. The pipeline worker runs at a fixed scale. If a user resizes a region on a 600px-wide canvas and the pipeline worker runs at a viewport width of 1200px, storing canvas coordinates will produce the wrong region on re-extraction.

The conversion is a simple ratio:

function canvasToWorker(canvasPx, canvasWidth, workerWidth) {
  return (canvasPx / canvasWidth) * workerWidth;
}

Every interaction, including resize, translate, and split line placement, converts to worker coordinates before storing. The canvas renders stored worker coordinates back to display coordinates on every draw. The canvas size at interaction time is the only thing that matters. Subsequent resizes do not corrupt stored corrections.

Eight-Handle Resize

Each selected region renders eight handles: four corners and four edge midpoints. The active handle is determined by which circle the pointer is nearest to. Cursor shape matches the resize direction:

const CURSORS = {
  nw: 'nw-resize', n: 'n-resize', ne: 'ne-resize',
  w:  'w-resize',                  e: 'e-resize',
  sw: 'sw-resize', s: 's-resize', se: 'se-resize',
  center: 'move'
};

On drag start, the code records the active handle and the pointer's starting position. On drag move, the delta from the start position is applied to the relevant edges of the bounding box. A 4px minimum constraint prevents collapsing a region to zero width or height.

Dragging the center handle translates the whole box. Dragging any edge handle adjusts only that edge.

Keyboard Reclassification

Pressing a letter key while a region is selected overrides its classification and marks it algorithm: 'custom-override'. The override is recognized by the extraction assembler and used instead of the pipeline's classification.

Delete does not erase the region. It marks skip: true. On re-extraction, any region marked skip: true is excluded from the output. The pipeline may re-classify the area differently on the next run. If the user wants a region permanently suppressed, reclassifying it to a benign type (box, image) is more stable than deleting it.

Column Split Bypass

The automatic column detector scans for X positions where the fewest text items overlap. It works well on standard two-column academic layouts. It fails on documents with narrow gutters, unusual column proportions, or strong visual dividers that the algorithm reads as content.

When the automatic detector produces wrong column boundaries, adding a manual split line overrides the detector entirely for that document:

if (manualSplits && manualSplits.length > 0) {
  const filtered = manualSplits
    .filter(s => s.x > vpWidth  0.05 && s.x < vpWidth  0.95)
    .sort((a, b) => a.x - b.x);
  return buildSplitsFromManual(filtered, vpWidth);
  // early return: no automatic scan runs
}

The split lines are drawn as dashed vertical markers on the canvas. A handle at the top allows drag repositioning. Double-click removes a line. The bypass is all-or-nothing: if any manual splits exist, the automatic scan is skipped entirely.

Tradeoffs

Corrections are session-local. They are not persisted. Re-opening a document starts fresh with automatic detection. This is the right tradeoff for a correction tool rather than an annotation layer: the pipeline should be good enough that re-correction is occasional, not routine.

The delete-as-skip pattern means the pipeline still processes the area on re-extraction. If the pipeline re-claims it differently after a delete, the user may need to reclassify rather than delete. This is an edge case in practice but worth knowing.

What to Watch For

All stored corrections are in worker coordinate space. The canvas-to-worker conversion uses the canvas width at the time of interaction. If the canvas is resized between interaction and re-extraction, the stored correction is still correct because it was already converted. The only time coordinates can go wrong is if the worker viewport size changes between document loads, which should not happen for the same document, but would require a re-correction if it did.

Read this post in the full Engineering Journal →