Postmortem: Three Bugs That Were All the Same Mistake

TLDR

During the implementation of our PDF processor observability panel, three distinct bugs surfaced: a worker thread hang during page message posting, phantom horizontal dividers inside paragraphs, and silent re-extraction failures. Despite different symptoms, all three shared one root cause: new observability code running alongside existing algorithms without respecting environmental constraints (serialization limits, stage execution order, and buffer transfer semantics).

Defect Incident	Observed Symptom	Underlying Environmental Violation	Structural Resolution
Worker Thread Hang	Extraction stalls indefinitely on image pages	Serializing large base64 image strings in postMessage	Omit base64 image data from manifest payload
Phantom Dividers	Paragraph copy split by false horizontal rules	Divider stage executed before text regions were populated	Re-ordered pipeline stage execution
Re-Extract Hang	Re-extraction button hangs after initial document load	ArrayBuffer neutering during sub-worker transfer	Execute bytes.slice() prior to transfer

---

Problem statement: three parallel failures

Adding an observability inspection layer to our document extractor triggered three independent defects:

1. Extraction Hang: Processing PDF pages containing large images froze the UI progress bar.
2. Document Corruption: Extracted HTML contained spurious <hr> horizontal divider tags inside body paragraphs.
3. Re-Extraction Freeze: Single-page re-extraction worked on initial page loads, but failed silently on subsequent re-runs.

---

Technical failure modes & incident analysis

Incident 1: structured clone payload bloat

• Root Cause: To populate region inspection previews, worker messages attached full base64-encoded image strings inside regions[].extractedImages.
• Failure Mode: For image-heavy PDF pages, serializing 8MB base64 strings per page via structured clone blocked the worker event loop, stalling main thread message handlers.
• Fix: Stripped base64 image payloads from region manifests, retaining only lightweight region IDs and bounding boxes.

Incident 2: out-of-order classifier execution

Root Cause: The divider line detector executed before* text paragraph classification. Failure Mode: The divider stage checked: "Is this horizontal vector line inside an existing text block?"* Because paragraph blocks had not been classified yet, the check evaluated to false, converting inline text underlines into document-level <hr> dividers.

• Fix: Re-ordered stage execution in orchestrator.js so line classification runs after text block boundaries are populated.

Incident 3: buffer neutering via worker transfers

• Root Cause: Passing input PDF Uint8Array data to PDF.js sub-workers triggered zero-copy transfers, neutering the underlying ArrayBuffer (byteLength === 0).
• Failure Mode: Worker code stored a direct reference to the input Uint8Array. Subsequent single-page re-extraction requests passed the neutered reference, causing PDF.js to throw unhandled internal errors.
• Fix: Sliced input byte arrays (bytes.slice()) before initiating initial PDF parsing.

---

The lessons & architecture principles

// REFACTORED WORKER INITIALIZATION PATTERN
self.onmessage = async (e) => {
  const { bytes, options } = e.data;
// 1. Slice bytes to preserve independent copy for re-extraction   const cachedBytes = bytes.slice();
// 2. Execute pipeline stages in correct dependency order   const pdfDoc = await pdfjsLib.getDocument({ data: bytes }).promise;   const pageRegions = await executeOrderedPipeline(pdfDoc, options);
// 3. Post lightweight manifest (No heavy base64 strings!)   self.postMessage({     type: 'page_complete',     regions: sanitizeManifestForTransfer(pageRegions)   }); };

Rule of thumb: When adding observability or inspection layers to working pipelines, audit payload serialization sizes, stage dependency execution order, and buffer transfer lifetimes.