Pdf Processor
Borrowed Math, Original Niches: What's New When the Algorithms Aren't
TLDR
Building a browser-native PDF extraction engine involves minimal mathematical innovation: lattice intersection algorithms, Y-band clustering, and XY-cut partitioning have existed in academic literature since the 1980s. Originality lies in architectural constraints: executing the entire pipeline client-side inside Web Workers on top of PDF.js, enforcing non-overlapping region invariants, and enabling per-page streaming.| Architectural Dimension | Traditional Python Extraction Tools | Client-Side Web Worker Pipeline |
|---|---|---|
| Execution Environment | Server-side Python runtime | Client-side Web Worker threads |
| Data Privacy | Requires uploading PDF bytes to server | 100% On-device processing (Zero server transit) |
| Memory Isolation | Single-threaded backend process | Nested Web Workers with page streaming |
What is borrowed vs. What is novel
Standard industry primitives (borrowed)
- Lattice Line Intersections: Projection profile algorithms (Tabula, Camelot, pdfplumber).
- Y-Band Text Clustering: Font-proportional line grouping (pdfminer).
- Spatial Indexing: Standard 2D Spatial Hash and KD-Tree lookups.
Architectural innovations (novel)
- Nested Web Worker Execution: Running spatial extraction inside dedicated Web Workers on top of PDF.js operator streams.
- Non-Overlapping Region Invariants: Tracking
assignedTextIndicesto prevent multi-pass classifiers from claiming identical text twice. - Viewport vs. Point Coordinate Discipline: Storing both PDF user points (
fontSize,width) and viewport pixels (vFont,vWidth) explicitly on every metadata record.
Technical architecture: non-overlapping text indices
Prevent multi-stage classifiers from double-claiming text using a shared assignedTextIndices bitset:
export class DocumentExtractionContext {
constructor(textItems) {
this.textItems = textItems;
this.assignedIndices = new Set();
}
// Retrieve only unclaimed text items for downstream classifiers getUnclaimedItems() { return this.textItems.filter((_, idx) => !this.assignedIndices.has(idx)); }
// Claim items once a classifier claims a region claimItems(claimedItemIndices) { claimedItemIndices.forEach(idx => this.assignedIndices.add(idx)); } }
By maintaining this invariant across stages (Lattice $\rightarrow$ Stream Table $\rightarrow$ Column Split $\rightarrow$ Paragraph), downstream detectors never receive items already owned by earlier stages.
Rule of thumb: Define product novelty by your architectural constraints and negative space, what you refuse to build, rather than expecting core mathematical primitives to be unique.
Read this post in the full Engineering Journal →