Your validation layer can run, pass CI, and check nothing

TLDR: A rule engine that looks up metadata by key is only as alive as that key's spelling. If the keys it queries do not match the keys the data actually carries, every rule evaluates to "nothing wrong" and stays green. Syntax checks and smoke tests both pass. The only way to catch it is to assert that a known-bad input produces a known finding.

The problem class

You have a validation layer. It walks a graph or a document, looks up per-element rules from a spec table, and reports violations. Think ESLint rule configs, a schema validator keyed by field name, a linter that reads type annotations, an electrical rule checker that reads pin roles.

The rules join two data sources: the thing being checked (which carries identifiers) and the spec that says what each identifier means. The whole system rests on those identifiers matching.

When they do not match, nothing crashes. The lookup returns undefined, the rule sees no metadata, and it concludes there is nothing to flag. Multiply that across every element and your validator reports a clean bill of health on genuinely broken input. It is the worst kind of dead code: the kind that looks alive.

The naive mental model

Here is the shape almost everyone writes first. A spec table keyed by human-friendly names:

const SPECS = {
  andGate: {
    pins: {
      IN1: { role: 'input' },
      IN2: { role: 'input' },
      OUT: { role: 'output' },
    },
  },
};

And a rule that reads a pin's role off that table:

function checkBusContention(net) {
  const outputs = net.pins
    .map(pin => SPECS[pin.component].pins[pin.id]?.role)
    .filter(role => role === 'output');
  if (outputs.length >= 2) {
    report(${net.id}: two outputs shorted together);
  }
}

Read it and it looks correct. Two output pins on one net is a short, and the rule reports it. Ship it.

Why it fails

The pin.id in the graph does not come from the spec. It comes from wherever the element was actually created. In our case the pins were emitted by a rendering kit that labeled them in1, in2, out, lowercase, generated positionally. The spec used IN1, IN2, OUT.

So SPECS['andGate'].pins['out'] is undefined. The optional chain swallows it. outputs is always empty. outputs.length >= 2 is never true. The rule runs on every net, forever, and never once reports the short it exists to catch.

The failure is invisible to the two checks people reach for. A syntax check sees valid JavaScript. A smoke test that draws a circuit and confirms it renders sees a circuit that renders. Neither asserts that a known violation produces a finding, because the whole point of a smoke test is that the happy path works.

We only found it by drawing two gate outputs into the same node on purpose and expecting an error. Nothing came back. That expectation is the entire test.

The better model: verify the join, not the parts

The fix in code was trivial: re-key the spec to the identifiers the data actually carries.

const SPECS = {
  andGate: {
    pins: {
      in1: { role: 'input' },
      in2: { role: 'input' },
      out: { role: 'output' },
    },
  },
};

But the trivial fix is not the lesson. The lesson is how to never ship this class again. Three things.

First, get the real identifiers from the runtime, not from your intent. We pulled the actual pin ids out of the live rendering kit:

kit.symbols.map(s => ({
  id: s.id,
  pins: [...s.svg.matchAll(/data-pin="([^"]*)"/g)].map(m => m[1]),
}));

That printed ['in1', 'in2', 'out'] and ['0', '1'] and ['base', 'collector', 'emitter']. The spec had guessed IN1/OUT, A/B, Base/Collector/Emitter. Every single one was wrong. Reading it from the source of truth would have caught all of them at authoring time.

Second, encode the invariant where it can be seen. We added a comment above the spec listing every real id the kit emits, and a one-line note that the keys are load-bearing for the rule engine. A convention that lives only in your head regresses the next time someone adds a component.

Third, and most important: write the test that asserts a violation. Not "does it run" but "does a known-bad input produce the finding."

test('bus contention fires on two shorted outputs', () => {
  const net = twoOutputsOnOneNet();
  expect(check(net)).toContainEqual(
    expect.objectContaining({ rule: 'bus-contention' })
  );
});

If that test had existed, the mismatch would have failed CI on day one. Its absence is why the rules could be 100% dead and 100% green at the same time.

Tradeoffs

Re-keying a spec to runtime identifiers couples your validation config to whatever emits those identifiers. If the kit renames a pin, the spec breaks. That coupling is real, and it is the correct coupling: the validator's job is to reason about the actual data, so it should break loudly when the actual data changes, not silently keep passing. A validator decoupled from reality is worse than no validator, because it manufactures false confidence.

The deeper tradeoff is philosophical. A lookup that returns undefined on a miss is ergonomic and forgiving, and that forgiveness is exactly what hid the bug. For a validation join, you want the opposite. Consider making a missing key throw, or logging every unresolved lookup during a test run, so a mismatch surfaces as noise instead of silence.

The one line to keep

A rule engine that runs is not a rule engine that fires. The only proof it works is a known-bad input that produces a known finding. Everything else, including green CI, is compatible with checking nothing.