Cache the Model, Replay the Assembler: Making Human Corrections Instant in an ML Pipeline
TLDR
If a user corrects your model's output and you re-run the model, you are paying for the expensive, non-deterministic pass to answer a question it already answered. Split the pipeline into a probabilistic stage (run once, cache the result) and a deterministic assembly stage (replay on every edit). Human corrections become constraints on the assembler, not new inputs to the model. Re-extraction goes from seconds to milliseconds, and it becomes reproducible.
The Problem Class
You have a pipeline where a machine-learning model produces a first guess and a human corrects it. Object detection with a bounding-box editor. Transcription with a text-correction UI. Layout analysis with a region editor. The pattern is everywhere: model proposes, human disposes.
The naive implementation re-runs the model when the human edits. The user drags a box, clicks "re-run," and waits while the GPU chews through inference again. This feels correct: the edit changed the input, so the pipeline should recompute.
It is the wrong reflex. The human's correction is not new input for the model. It is a correction of the model. Feeding it back through the model asks the same non-deterministic function the same question and hopes for a better answer. What you want is to keep the model's output fixed and change only how you use it.
The fix is a pipeline split: two stages with very different cost and determinism profiles.
The Two Stages
Consider a concrete case: extracting structured content from a scanned page. Two models do the heavy lifting.
A layout detector finds regions and labels them: table, figure, heading, paragraph. A text recognizer reads the pixels into words, each with a bounding box and a confidence score.
Neither output is the final result. They are raw materials. A third stage, the assembler, takes detected regions plus recognized words and reconstructs the document: it groups words into columns, snaps them into table cells, orders them into a reading flow, and emits HTML.
Here is the cost and determinism table that drives the whole design:
| Stage | Deterministic? | Cost | Depends on user edits? | |---|---|---|---| | Layout detection (ML) | No | High (per-page inference) | No | | Text recognition (ML) | No | High (per-page inference) | No | | Assembly (geometry) | Yes | Low (milliseconds) | Yes |
The ML stages are expensive and non-deterministic. The assembly stage is cheap and deterministic. The user's corrections (split this column, this region is actually a table, ignore this detection) only ever affect assembly. They never change what the models saw.
That table is the entire argument. Once you see it, the architecture writes itself.
Run the Model Once, Cache Its Output
The first stage runs a single time, when the document loads. It renders each page, runs detection and recognition, and then does one crucial thing: it converts the model output into a normalized, model-agnostic intermediate representation and caches it.
// One-time pass: render page, run both models, normalize, cache.
const [regions, ocr] = await Promise.all([
layoutDetect(pageBitmap), // [{ label, bbox, confidence }]
recognizePage(pageCanvas), // { words: [{ text, bbox, confidence }] }
]);
// Normalize OCR words into the SAME shape a native text layer would produce, // so the deterministic assembler cannot tell scanned from vector input. const textItems = synthesizeTextItems(ocr.words);
// Cache the synthetic inputs keyed by page. This is the whole trick. cache.set(pageNum, { textItems, regions, filledRects, viewport });
The key decision is the normalization. The OCR words are reshaped into the exact structure a page with a real embedded text layer would have. From the assembler's point of view, a scanned page and a born-digital page are now indistinguishable, so one assembler serves both paths and the model output is just data in a map.
Notice what is not cached: nothing about how to group the words. No columns, no tables, no reading order. Those are the assembler's job, and it has not run yet.
Replay the Assembler on Every Edit
The second stage is a pure function of the cached data plus a small pipeline-config object that carries the user's corrections.
function reprocessPage(pageNum, corrections) {
const cached = cache.get(pageNum);
if (!cached) return; // model never ran for this page
// Same cached model output every time. Only corrections changes. const { regions, textMeta, columnSplits } = classify( cached.textItems, cached.regions, cached.viewport, { skip: corrections.skip, // ignore region types manualSplits: corrections.manualSplits, // forced column boundaries customRegions: corrections.customRegions, // user-drawn overrides scaleOverrides: corrections.scaleOverrides, // re-tolerance the geometry }, );
return assemble(regions, textMeta, cached.textItems, cached.viewport); }
Every user gesture maps to one field in that config:
- Drawing a column split adds a forced boundary at an x-coordinate. The same OCR words re-flow into two columns.
- Selecting a region the detector missed injects a
customRegionwith an override flag into the region list the assembler builds from. - Toggling off a region type adds it to
skip, and the assembler drops it. - Nudging a confidence slider changes
scaleOverrides, which re-tolerances how aggressively geometry snaps.
classify and assemble, not a second inference pass.
Why Determinism Is the Real Payoff
Speed is the obvious win. The subtle one is reproducibility.
A deterministic assembly stage means the same cached input plus the same corrections always produces the same output. You can serialize the corrections, store them next to the document, and reconstruct the exact extraction later without re-running any model. The corrections are the edit history, and they are tiny: a few x-coordinates and a couple of region overrides, not megabytes of re-inferred boxes.
It also makes corrections auditable. Each override is an explicit, inspectable claim ("there is a column boundary here") on top of a probabilistic guess, and because the function is deterministic you can diff its runs to show exactly which correction changed the output. A model re-run gives you none of that; its output can drift for reasons unrelated to the edit.
Stated plainly: the model drives the first guess, and the deterministic engine is the worker that produces the artifact under human supervision. Keeping the model out of the correction loop is what makes the corrections trustworthy.
The Tradeoff You Are Accepting
This split draws a hard line: corrections can only change layout, never the model's content. If the text recognizer misread a glyph, reading "rn" as "m", no amount of column-splitting fixes it, because the cached word is wrong and the assembler faithfully places the wrong word. Layout tools reshape grouping; they cannot re-transcribe.
Surface that limitation honestly in the UI so users do not fight the wrong tool. If content-level correction matters, it needs its own path: an editable text overlay, or a targeted re-run of the model on just the corrected crop, written back into the cache. Both are additive. Neither abandons the split, because both are just a second, narrower write into the same cached intermediate.
The point is that content correction is the exception. Most "the model got it wrong" cases are grouping errors: merged columns, missed tables, wrong reading order. Those are exactly what the cheap deterministic replay handles for free.
The Generalizable Lesson
When a human corrects machine-learning output, ask what the correction actually constrains. Almost always it constrains a downstream deterministic step, not the model itself.
So run the expensive non-deterministic pass once, cache its result in a normalized shape, and express every human correction as a constraint on a deterministic function of that cache. You get instant re-runs, reproducible edits, an auditable correction history, and a clean seam where content-level fixes can be added later.
The reflex to re-run the model on every edit is the mistake. The model already answered. Change how you use the answer, not the question you ask it.