Post-Mortem: The 8GB Memory Spike and the Silent Web Worker Serialization Bug

TLDR

During the integration of the headless batch pipeline into the Ginexys navigation panel, two critical defects surfaced: Web Worker extractions silently returned empty HTML payloads due to a message payload key mismatch (buffer vs bytes), and focusing batch document cards triggered redundant main-thread re-extractions that pushed browser memory usage to ~8GB. We resolved both defects by unifying worker serialization keys and building a zero-re-extraction cached memory mount pipeline.

Defect Area	Root Cause	Engineering Solution
Silent Worker Extraction Failure	Worker expected e.data.bytes, pool sent e.data.buffer	Unify payload destructuring to check both buffer and bytes
8GB Browser Memory Spike	Card click invoked handleFile() re-extraction loop	Direct memory mount from cached BatchQueueItem IR
Detached ArrayBuffer Loss	Worker message return omitted transferable ArrayBuffer	Post back bytes: bytes.buffer as transferable object

---

Technical problem analysis & solutions

1. The silent serialization key mismatch

When WorkerPool dispatched batch items, it passed the document payload under the property buffer:

// WORKER POOL DISPATCH
workerEntry.worker.postMessage({
    type: 'process_batch_item',
    id: task.item.id,
    buffer: arrayBuffer
}, [arrayBuffer]);

However, inside pipelineWorker.js, the message listener destructured bytes instead of buffer:

// DEFECTIVE WORKER LISTENER
self.onmessage = async (e) => {
    if (e.data.type === 'process_batch_item') {
        const { id, bytes, format } = e.data; // DEFECT: bytes was undefined!
        
        // TextDecoder received undefined, producing empty string output
        const rawText = new TextDecoder().decode(bytes); 
        self.postMessage({ type: 'batch_item_complete', id, result: { extractedHTML: rawText } });
    }
};

Because bytes was undefined, text decoding and PDF parsing silently failed and posted empty strings back to the queue.

Remediation:

Support both ArrayBuffer transfers and raw byte arrays in the worker listener:

// REFACTORED WORKER LISTENER
self.onmessage = async (e) => {
    if (e.data.type === 'process_batch_item') {
        const { id, buffer, bytes: rawBytes, format } = e.data;
        const bytes = rawBytes ? new Uint8Array(rawBytes) : (buffer ? new Uint8Array(buffer) : null);
        
        // Process valid Uint8Array
        let extractedHTML = parseDocument(bytes, format);
self.postMessage({             type: 'batch_item_complete',             id,             result: {                 extractedHTML,                 bytes: bytes ? bytes.buffer : null             }         }, bytes ? [bytes.buffer] : []);     } };

---

2. Eliminating focus-triggered re-extraction loops

To display an extracted batch item when clicked, focusBatchItem initially called loadFileToSlot(). This re-read raw files and re-spawned geometry parsing workers on every click. Switching between 5 uploaded files resulted in 5-10 concurrent parsing pipelines, spiking browser memory to ~8GB.

Remediation:

Mount pre-extracted results from memory instantly without invoking file loading handlers:

// REFACTORED: Instant cached mount
export async function focusBatchItem(itemId, slotNum = 1) {
    const item = batchQueue.getItem(itemId);
    if (!item || item.status !== 'completed') return;
const slot = slotNum === 1 ? state.pdf1 : state.pdf2;
// Read directly from cached batch result object     slot.extractedHTML = item.extractedHTML || '';     slot.bytes = item.bytes || null;     slot.gxDoc = item.gxDoc || null;
// Hydrate DOM in <10ms without worker invocation     applyHtmlEverywhere(slot.extractedHTML, null);     if (item.format === 'pdf' && item.bytes) {         await renderPDFToCanvas(item.bytes);     } }

Rule of thumb: Always verify serialization key parity across Web Worker boundaries, and never invoke heavy extraction pipelines when pre-computed IR data already exists in memory.