Stop publishing coverage percentages nobody can check

TLDR: Our netlist first reported "115 unnamed components". There were no unnamed components. The denominator was clusters of geometry, most of which are junction dots. A coverage number whose denominator you cannot inspect is decoration.

The position

If you report that your extractor achieved some fraction, the reader's only useful question is "out of what". Most tools cannot answer, because the denominator is an internal artefact.

Report the evidence instead. How many candidates existed, how many were consumed, and which ones were not. That is checkable against the source document, and checkable is the only property that makes a completeness claim worth printing.

Why the common pattern exists

Because a percentage is a great summary and a list is not, and because during development the denominator genuinely is the internal count. You are looking at 150 clusters and 35 named ones and the ratio tells you something real about the code you just wrote.

It stops being real the moment it leaves your machine, because your user has no idea what a cluster is.

Why it fails

We shipped components: symbols.length - named.length, which came out as 115.

Read as a user would: the tool found 150 components and could only name 35 of them. Terrible. Someone would reasonably conclude the naming is broken.

The truth: the drawing contains 36 component designators. We resolved 35. The other 115 clusters are junction dots, pin stubs, and arrowheads, which are geometry and were never parts.

So the number was not merely imprecise. It pointed at the wrong subsystem. Anyone acting on it would have gone to improve the label matching, which was already operating at 97% of the available evidence, and left the actual limitation untouched.

The actual limitation is that you cannot name parts the drawing never labelled. That is a ceiling, not a bug, and no amount of work moves it.

The better shape

COVERAGE
  designators in document : 36
  designators resolved    : 35
  unresolved              : L13
  nets with no label      : 2

Every line of that is verifiable by opening the PDF. A reader can search for L13, find it, see why it was missed, and tell us. That is a bug report we can act on, generated by the user, from a number we published.

The same applies to inferences the tool made along the way. Ours has to decide which stroke colours carry nets and which carry symbol outlines, which it does by measuring how far connected runs span. So it prints that too:

STROKE LAYERS (net vs symbol, by median connected-run span)
  rgb(80,80,80)     59 segs  median run   3.7pt  -> symbol
  rgb(128,0,0)     887 segs  median run     8pt  -> symbol
  rgb(0,128,0)     326 segs  median run  57.6pt  -> net

If the tool got that backwards, the reader can see exactly what it believed and why. Without it, a wrong netlist is inexplicable.

What you give up

A clean headline. "97% coverage" fits in a badge. Four lines of evidence do not.

Flattering numbers. Ours went from an implied 23% to an explicit 35 of 36, which is better, but the direction is not guaranteed. Sometimes honest denominators are worse than the internal ones, and that is the point.

Some output size. A few lines per document, and a list that grows with the failures. If the unresolved list is long, that is information, not a formatting problem.

When a percentage is fine

When the denominator is something the user supplied and can count: files processed, rows imported, tests run, pages converted. Then the fraction is grounded in their input and they can check it.

The test I would apply before printing any completeness number: can the reader determine the denominator without reading our source code? If not, print the evidence and let them do the arithmetic.