One typed model between import and export
TLDR
We grew four import formats and five export formats one at a time, and each pair wanted its own converter. We used our rendered HTML as the de facto pivot until adding DOCX import proved the DOM only stores what it draws. We replaced that with a typed intermediate representation every importer emits and every exporter reads. The N x M converter matrix became N plus M, and the old HTML path stayed behind as a fallback, so the change shipped without a rewrite.| Pivot strategy | Number of converters | Structure fidelity | Cost to add a format |
|---|---|---|---|
| Direct N x M converters | N x M | Inconsistent per pair | New exporter for every pair |
| Rendered HTML re-parse | N + M | View projection only | Re-infer structure from a view |
| Typed IR pivot | N + M | Full schema, lossless by contract | One importer, zero exporter changes |
The converter matrix problem
We build a browser-based PDF extraction tool. It started as a PDF importer with an HTML export. Then Markdown import. Then HTML import. Then someone asked for XML, DOC, and JSON out, and eventually DOCX in.
Each pair looked easy in isolation. "Markdown export just walks the DOM." "JSON export just serializes the text." Nobody planned the matrix. We tripped over it when the same facts, "this is a header row", "this is a two-column layout", "this text is bold", were being re-derived in four different places, each with its own heuristics.
Why we tried rendered HTML as the pivot
The rendered DOM was already there, it rendered correctly, and our extractor stamped structure into class names and attributes:
<div class="pdf-page-row" data-zones='{"cols":2}'>
<div class="pdf-col pdf-col--left" data-col-id="col-1">...</div>
<div class="pdf-col pdf-col--right" data-col-id="col-2">...</div>
</div>
Exporters read structure from those markers. New importers produced more of the same HTML. It felt like N plus M was already true. It was, until a format came along whose structure the DOM never carried.
Where the DOM leaks structure
Three leaks convinced us the DOM is a projection, not a model.
- The view only stores what it draws. A column split ratio rendered as
grid-template-columns: 40% 60%has no data slot. It survives rendering and dies in the round trip. We proved this with our own round-trip test: the HTML went to the model and back, blocks matched, and the ratio silently fell back to equal columns. - Re-parsing re-implements the extractor. Reading "is this row a header" from the DOM means writing detection heuristics in the exporter, duplicating the ones the extractor already ran. Two copies of a heuristic disagree exactly the way heuristics disagree.
- New formats expose the gap. DOCX brings numbered lists, merged cells, explicit widths. Re-inferring those from a rendered view is worse than not supporting them.
The model: one schema, two roles
We defined a typed, serializable document model. Every importer produces it. Every exporter consumes it. No converter reads a view.
The schema is a shape, not a policy. It describes what a document is, not how any format decided it. That distinction matters, because it is what makes the model safe to own.
export const GX_DOC_SCHEMA = 'gx-doc/1';
export function createDoc(meta = {}) { return { schema: GX_DOC_SCHEMA, meta: { source: meta.source ?? null, title: meta.title ?? null, pageCount: meta.pageCount ?? null, }, pages: [], }; }
Blocks are typed, not stringly: heading, paragraph, table, list, image, callout, divider. Text carries optional runs for bold and italic. Tables carry headers, rows, flags, and a confidence score. Layout lives in zones with a column count and a per-block column index, so multi-column shape survives import and re-render.
Validation is pure and cheap. We built a test harness around it, and the output is the contract:
validate: {"ok":true,"errors":[]}
pages: 1 blocks: 3 types: heading,paragraph,table
That is the whole argument in one line. Exporters can assume the shape is right instead of defending against the DOM.
Coexist, do not cut over
The model did not require a rewrite. The rendered HTML stayed as the render cache and the fallback for documents that predate the IR:
const gxDoc = pdf?.gxDoc || null;
if (gxDoc) {
tables = gxDoc.pages.flatMap(readTables);
} else {
tables = [...dom.querySelectorAll('table')].map(describeTable);
}
New code path first, old path preserved, not deleted. The IR ships behind real documents, builds stay green, and the fallback retires only when nothing produces a document without it.
What it cost us
The model is not free.
- Schema versioning.
gx-doc/1is a string in the payload. Adding a field is easy; changing one means a version bump and a migration decision. - Deciding what earns a slot. Every field is a promise that all importers fill it and all exporters read it. The discipline is to add fields when a real format needs them, not when they feel useful.
- Dual paths during migration. Two code paths to keep honest until the fallback retires. A bug in either shows up as a wrong export, so the fallback had to stay tested, not just present.
โ built in 26.28s
main-*.js 4,446.02 kB โ gzip: 1,194.18 kB
Rule of thumb
If a value must survive a round trip, give it a named slot in a typed model. If it only affects rendering, let the view own it. Never let exporters re-derive from a view what an importer already decided.
When the next format request arrives, the answer is "add an importer", not "add four converters".