Your document parser's type set is the product

TLDR: A document parser can only preserve what its intermediate representation has a name for. If the IR has paragraph but not equation, every equation in every document becomes a paragraph on the way in and stays one forever. The type set is not schema housekeeping, it is a hard ceiling on the product.

The problem class

You are building anything that reads structured documents: a PDF extractor, a DOCX importer, a scraper that normalizes HTML into something queryable. Between the parser and every consumer sits an intermediate representation. Everyone treats the IR as plumbing.

It is not plumbing. It is the contract that decides what the rest of the system is allowed to know.

Here is the failure in its purest form. Two academic PDFs go through a pipeline that classifies page regions geometrically: tables, paragraphs, headings, images, boxes, dividers, headers, footers. Both documents are mostly equations and citations. The extraction reports 300+ regions per document and looks like a success.

Then you ask the system a question it should be able to answer: "give me the equations." It cannot. Not because detection failed, but because there is no such thing as an equation in the type set. Every one of them is a paragraph. The bibliography is 156 entries of paragraph.

The naive approach

The naive move is to treat the missing types as a detection problem. Write a better classifier, tune a threshold, add a model. That is the wrong end of the pipe.

Detection is downstream of naming. Until equation exists as a type, a perfect detector has nowhere to put its answer. You would be writing a function whose return value the system cannot represent.

The second naive move is to smuggle the information in as a flag on an existing type:

{ type: 'paragraph', isEquation: true, latex: '...' }

This looks pragmatic and it rots quickly. Every consumer now has to know that a paragraph might not be a paragraph. The renderer needs a branch. The exporter needs a branch. The validator cannot check the invariant that an equation must carry content, because there is no equation to check. You have added a type without adding a type, and paid the cost of both.

The better model

Treat the type set as closed, enumerated, and validated. Adding a legend is an explicit act with a defined blast radius, and every layer is required to handle every member.

const BLOCK_TYPES = new Set([
    'heading', 'paragraph', 'table', 'list', 'image', 'callout', 'divider',
    'equation', 'reference',
]);
// A text block additionally carries a role, which is what makes page // furniture separable from body prose instead of all three arriving // as the same untyped paragraph. const TEXT_ROLES = new Set(['header', 'body', 'footer']);

Two decisions in there are worth pulling out.

Equations store their source, not their rendering. The typeset markup is a view and is re-derivable. The TeX is the content:

case 'equation': {
    const tex = esc(block.latex || block.text || '');
    return <p class="math-block" data-latex="${tex}">${tex}</p>;
}

Store the rendering instead and a round trip destroys the equation. That is not hypothetical: before this branch existed, the importer read the rendered glyph soup back as prose, so every export-then-import cycle turned an equation into gibberish that happened to look like text.

References store entries, not a blob. A bibliography that arrives as one string is not something anyone can cite, count, reorder, or export. The type carries a list:

{ type: 'reference', entries: ['Anderson, P. W., …', 'Barends, R., …'] }

Role is a field, not three types. A running head and a body paragraph are both paragraphs. They flow the same way, they carry the same runs, they belong to the same destination. What differs is what they are for. Making them three sibling types duplicates every paragraph code path three times; making role a field on one type keeps one path and still lets a consumer select or ignore furniture as a set.

Making the type set enforce itself

A closed set is only worth having if something checks it. The validator is where the ceiling becomes real:

if (block.role != null && !TEXT_ROLES.has(block.role)) {
    errors.push(block ${bi} has unknown role "${block.role}");
}
if (block.type === 'equation' && !String(block.latex || block.text || '').trim()) {
    errors.push(block ${bi} is an equation with no latex or text);
}
if (block.type === 'reference' && !(block.entries || []).length) {
    errors.push(block ${bi} is a reference block with no entries);
}

These are not style checks. They are the statements that make the type mean something: an equation without content is not an equation, it is a bug that would otherwise travel silently to whatever consumed it.

The mapping layers are where types earn their keep

Once the type exists, every layer needs one line, and each line is trivial:

const BLOCK_TYPE_TO_REGION = {
    heading: 'HEADING', paragraph: 'PARAGRAPH', list: 'LIST',
    image: 'IMAGE', callout: 'BOX', divider: 'DIVIDER',
    equation: 'MATH', reference: 'REFERENCE',
};
const ROLE_TO_REGION = { header: 'HEADER', footer: 'FOOTER' };

That triviality is the point. The expensive part was never the mapping, it was that the destination had no slot. Add the slot and the work collapses into table lookups that a reviewer can verify at a glance.

Tradeoffs

A closed type set has real costs and you should take them deliberately.

Every new type touches every layer. Adding equation and reference meant editing the validator, the renderer, the importer, the region mapper, the classifier, and the UI legend. That is six files for two nouns. The alternative is an open type set where a consumer receives something it has never heard of and guesses.

Types you add are types you cannot cheaply remove. Exports written today carry them. Pick names for what the thing is in the document, not for how you currently detect it: reference survives a rewrite of the reference detector, regex-matched-citation-block does not.

A type with no producer is a lie. If you add equation and nothing ever emits one, the UI shows an empty tab and users conclude the document has no equations. Either ship the detector alongside the type or do not ship the type.

The lesson

Before tuning a detector, check whether the answer has anywhere to land. Enumerate the things your documents actually contain, then check that list against your IR's type set. Everything on the first list that is missing from the second is currently being silently destroyed, and no amount of detection accuracy will save it.