Confidence Scores Are the Wrong Default for Pipeline Observability

TLDR

Defaulting to abstract confidence score sliders (0.0 to 1.0) in deterministic geometric processing tools creates opaque user controls. A confidence score of 0.63 offers no actionable intuition on how to adjust parameters. In contrast, exposing physical structural thresholds (e.g., column gap distance, Y-band line height) paired with live canvas ghost overlays gives users clear visual feedback on how adjustments impact document extraction.

Observability Control	Parameter Type	User Mental Model	Canvas Visualization
Abstract Confidence Score	Downstream composite ratio	Opaque ("What does 0.63 mean?")	Abstract pass/fail color toggles
Structural Geometry Threshold	Physical pixel / ratio bounds	Intuitive ("Merge gap at 15px")	Live geometric ghost overlay bands

---

Problem statement: the fallacy of opaque confidence sliders

When designing pipeline observability panels, developers often expose internal confidence scores:

// Downstream composite score
const confidenceScore = (colAlignmentScore + rowSpacingScore) / 2;

If a user drags a confidence slider from 0.60 to 0.75 and a table disappears, they cannot determine why the table failed validation.

Did column alignment variance fail? Did row spacing exceed tolerances? Abstract confidence sliders disguise geometric cause-and-effect behind opaque floating-point numbers.

---

Technical architecture: geometric input thresholds

Instead of exposing downstream confidence scores as primary sliders, expose physical input thresholds:

// Structural geometric input thresholds
const PIPELINE_THRESHOLDS = {
  R_Y_BAND: 0.35,     // Line-grouping Y-band tolerance ratio
  R_COL_TOL: 1.20,    // Column anchor clustering radius ratio
  R_STREAM_GAP: 2.50  // Section break gap ratio
};

Live ghost overlay feedback

Pair structural thresholds with live canvas ghost overlays so users see physical geometry change as sliders drag:

// Render interactive column anchor bounds on canvas preview
function drawColumnAnchorGhosts(canvasCtx, columnAnchors, rColTol, baseFontSize, scaleFactor) {
  const clusterWidthPx = baseFontSize  rColTol  scaleFactor;
canvasCtx.fillStyle = 'rgba(56, 189, 248, 0.2)';   canvasCtx.strokeStyle = 'rgba(56, 189, 248, 0.8)';
columnAnchors.forEach(anchor => {     const anchorX = anchor.centerX * scaleFactor;          // Draw column cluster tolerance region     canvasCtx.fillRect(anchorX - clusterWidthPx / 2, 0, clusterWidthPx, canvasCtx.canvas.height);     canvasCtx.strokeRect(anchorX - clusterWidthPx / 2, 0, clusterWidthPx, canvasCtx.canvas.height);   }); }

As users drag R_COL_TOL, column cluster regions expand visually on screen. Users immediately understand why adjacent text columns merge or separate.

Rule of thumb: Expose physical geometric input thresholds rather than downstream confidence scores in deterministic extraction pipelines. Use live ghost overlays to visualize parameter boundaries.