You built the hard half of provenance and shipped none of it
TLDR: We had a working lineage writer for months. Every cross-tool handoff carried an ordered chain of which tool touched a value, at what stage, with what score. Nothing in the product ever displayed it. A trail nobody can inspect has exactly the same observable behaviour as no trail at all.
The asymmetry
Write paths for provenance are genuinely hard. You need a stable record shape every tool agrees on, a rule for inheriting an incoming chain and appending to it, a place to hang the chain that survives serialization, and a discipline that stops any tool from dropping it.
Read paths are a list.
So the work naturally goes write-first, and then the read path never gets prioritised, because the writing is done and the feature "exists". It does not exist. Nobody outside the codebase can tell the difference between a system that records lineage perfectly and one that records nothing, if neither shows you anything.
How you notice
We noticed by auditing our own paid tier and trying to assign a completion percentage to each feature.
Every other line item had a number we could argue about. Provenance had a number and a note that said the number was meaningless, because the thing it measured could not be demonstrated. You cannot sell, demo, or QA a property with no observable behaviour. You cannot even confirm it works.
That last part matters most. A write-only feature is not just unsellable, it is untested in the way that counts. Unit tests confirm the record has the right fields. They do not confirm the chain is complete across three tools, because nothing assembles the chain to look at.
Where the reader goes
The instinct is to put it in each tool: every tool knows the handoffs it took part in, so let each one show its own.
That is wrong for the same reason per-tool capture was fragile. A tool only sees the hops it is party to. A chain that runs PDF to TAFNE to Schema is one object, and no participant holds all of it.
Put the ledger at the point every handoff passes through. In our shell that is one function in the kernel, _deliverToApp, which every cross-tool envelope goes through on its way to a receiving iframe. Recording there makes the ledger complete by construction rather than by every tool remembering to report:
if (window.GxLineage) {
try { window.GxLineage.record(deliverEnvelope, targetAppId); }
catch (e) { console.warn('[CWS Kernel] lineage record failed:', e); }
}
Nine lines including the guard. That is the whole collection layer, because the chains were already being assembled correctly by the writer.
Snapshot, do not alias
The one non-obvious correctness rule. An envelope is handed to a receiving tool that can mutate it, so a ledger that stores a reference to the chain silently changes after the fact:
chain: chain.map(function (r) { return Object.assign({}, r); }),
A ledger that can change retroactively is worse than no ledger, because it looks authoritative. We have a test that mutates the source envelope after recording and asserts the stored entry is unchanged.
The distinction the UI has to make
Two states look identical in a list and mean opposite things:
- a handoff whose chain is empty: the sending tool stamped nothing. That is a bug in the writer.
- a handoff whose chain has one entry: this is the first hop of a healthy trail.
No lineage on this handoff — the sending tool recorded nothing. The data arrived; its history did not.
This is the same principle as reporting null rather than 0 for a check that could not run. The expensive failures in verification tooling are always the ones where "nothing found" and "did not look" render the same.
What the panel needs to show
Not just the tools. Someone reading a lineage is asking a specific question, and it is usually one of:
Which tool produced this, and at what stage? Tool name plus stage, in order, numbered, with a connector drawn between hops so the chain reads as a path rather than a list.
How confident was each step? The score the producing tool self-reported. Ours renders it as a chip beside the stage, because a 0.82 extraction feeding a 1.0 validation is a completely different story from the reverse.
What did each tool actually do? The ordered ops. extract · tag then clean · verify tells you more than either tool's name.
Where did it start? The source artifact. A chain with no root is a chain you cannot check against anything.
And it has to be copyable as text. A bug report pasted into an issue should carry the trail the reporter was looking at, not a screenshot of it. toText() renders the same content the panel does, which also means the two cannot drift.
The summary before the detail
Above the entries, a chip per tool with a count of how many handoffs it produced. That is the question people actually ask first, "is this thing recording anything at all, and from where", and it answers it in one line before anyone reads a chain.
The general rule
If you have built recording without reading, you have built a liability, not a feature. The data accumulates, the code has to be maintained, the shape has to stay compatible, and none of it produces value or even confirms itself.
Before merging the writer, ask what surface will read it and when. If the answer is "later", the honest status is not "provenance: done, panel: todo". It is "provenance: not shipped".