Cache the Model, Replay the Assembler: Making Human Corrections Instant in an ML Pipeline

TLDR

Re-running expensive ML layout detection models (bounding box detectors, OCR engine passes) when users make visual corrections (adjusting column boundaries, overriding region types) introduces multi-second UI latency. Splitting the processing pipeline into two decoupled tiers, a probabilistic ML model tier (run once, cache output) and a deterministic geometric assembly tier (replayed instantly on user edits), reduces re-extraction latency from seconds to sub-milliseconds.

Processing Tier	Determinism	Computation Cost	Re-Execution Trigger
Probabilistic ML Tier	Non-deterministic	High (Seconds / GPU pass)	Document load only (Run once & cache)
Deterministic Assembler	100% Deterministic	Low (<5 Milliseconds)	User UI interaction / parameter edit

---

Problem statement: the ML re-execution anti-pattern

In document processing applications, machine-learning models provide initial predictions (layout region boxes, OCR glyph bounds).

When a user modifies a region boundary on screen and clicks "Apply", re-invoking the ML model creates severe latency issues:

1. High Latency: Users wait 2 to 5 seconds while GPU/Wasm inference re-evaluates the page.
2. Model Drift: Non-deterministic inference passes can return slightly altered bounding boxes for untouched regions elsewhere on the page.

---

Technical architecture: decoupled cache & assembler replay

[PDF Document] ---> [ML Inference Tier (Run Once)] ---> [Normalized In-Memory Cache]
                                                                |
[User Interactive Edits] --------------------------------> [Deterministic Replay Assembler (<5ms)] ---> [Updated HTML/DOM]

Step 1: one-time model execution & cache normalization

Run layout detection and OCR passes once on document import, normalizing model predictions into a standardized internal representation:

// ONE-TIME PASS: Execute inference, normalize shapes, and store in-memory cache
export async function initializeDocumentPageCache(pageCanvas, pageNum) {
  const [layoutRegions, ocrResults] = await Promise.all([
    runLayoutDetectionModel(pageCanvas),
    runOcrRecognitionModel(pageCanvas)
  ]);
// Normalize OCR outputs to match native vector text item format   const normalizedTextItems = ocrResults.words.map((word, idx) => ({     id: ocr_word_${pageNum}_${idx},     text: word.text,     vx: word.bbox.x,     vy: word.bbox.y,     vWidth: word.bbox.width,     vHeight: word.bbox.height,     confidence: word.confidence   }));
// Store in cache map. ML models will NEVER run for this page again.   pageModelCache.set(pageNum, {     textItems: normalizedTextItems,     detectedRegions: layoutRegions,     viewport: { width: pageCanvas.width, height: pageCanvas.height }   }); }

Step 2: pure deterministic replay assembler

Execute layout re-assembly as a pure function of cached model data and user overrides:

// FAST REPLAY PASS: Re-runs in under 5ms on user edits
export function reprocessPageWithCorrections(pageNum, userCorrections = {}) {
  const cachedData = pageModelCache.get(pageNum);
  if (!cachedData) return null;
// Apply user corrections (custom splits, region overrides, skip sets)   const classification = classifyTextAndLayout(     cachedData.textItems,     cachedData.detectedRegions,     cachedData.viewport,     {       skipTypes: userCorrections.skipTypes || new Set(),       manualSplits: userCorrections.manualSplits || [],       customOverrides: userCorrections.customOverrides || []     }   );
// Pure geometric assembly emits updated HTML fragment   return assemblePageHtml(classification.regions, classification.textMeta, cachedData.viewport); }

Rule of thumb: Run expensive non-deterministic ML models once and cache normalized predictions. Treat human edits as constraints on a fast deterministic assembly engine.