LLMs Should Propose Document Patches, Not Rewrite Documents

The primary reader is a developer adding an LLM to a document pipeline. The position is simple: if the document matters, the model should propose small typed operations rather than generate the final representation.

What the industry often does

The common pattern sends OCR text, layout boxes, or page images to a model and asks for clean Markdown or JSON. It is attractive because one prompt appears to replace reading order, hierarchy, table cleanup, and form interpretation.

For demos, it works. For production extraction, it destroys the ability to distinguish source recovery from semantic invention.

Why it fails

A rewritten document has weak identity. When a paragraph moves, was reading order fixed or was text paraphrased? When a table value changes, was OCR corrected or was a plausible number invented? String diffs cannot answer reliably, and rerunning a model may produce a different document.

Large prompts create another failure mode. Most pages are already resolved correctly by native PDF evidence and geometry. Sending everything asks the model to reconsider facts that were never ambiguous, increasing cost and the surface for accidental changes.

The better approach

Build a canonical graph first. Run deterministic checks for uncovered text, incompatible overlaps, weak tables, ambiguous order, and unmatched labels. Send only the failing subgraph to the model with a closed operation vocabulary:

{
  "op": "set_parent",
  "childId": "p8:caption:2",
  "parentId": "p8:picture:1"
}

The application validates IDs, types, containment, cycles, coverage, and whether the operation improves a named quality signal. The original evidence is immutable, and the patch carries before/after state for rollback.

This arrangement uses the model for what it does well: choosing among plausible semantic interpretations. Deterministic software retains authority over what actually changes.

What you give up

Typed operations require schemas, validators, stable IDs, and graph-aware tooling. The model cannot solve an entirely new problem until the operation vocabulary supports it. Some valuable repairs will be rejected because constraints are conservative.

Those are healthy limitations for consequential documents. A rejected suggestion can be reviewed. An elegant but fabricated final document may be impossible to diagnose.

When direct generation is right

Direct Markdown generation is reasonable for disposable summaries, personal notes, and workflows where the source remains nearby and exact fidelity does not matter. It is also a useful fallback when the only goal is human reading rather than data extraction.

It is the wrong authority for financial tables, forms, technical procedures, contracts, or any workflow that promises provenance.

The better question is not whether an LLM can restructure a document. It can. The question is whether the system can prove what changed, why it changed, and which source evidence survived. Constrained patches make that proof possible.