When the job is green but the ledger is wrong

Exit codes lie. How reconciliation assertions catch silent ETL truncation before finance notices.

Orchestrators love exit codes. A transform that catches exceptions, writes a partial file, and returns zero looks healthy in the UI while finance stares at a ledger that no longer ties to shipments.

The pattern we keep seeing

Error rows land in a side table or a “quarantine” path that nobody pages on. Volume grows slowly. Dashboards still paint last night’s success rate in comforting colors because they measure job completion, not business reconciliation.

A practical countermeasure

Pick the three feeds that would embarrass you in a leadership meeting. For each, define an expected relationship: row counts between extract and load, sum of a money field, or a distinct key count against an upstream authority. Fail the pipeline — or at least page a human — when the relationship breaks beyond a negotiated tolerance.

Tolerance is a product decision

Zero tolerance sounds virtuous and creates alert fatigue. Work with finance or operations to set bands that reflect known late arrivals and partial days. Document the band next to the job definition so the next engineer does not “tighten it for quality” at 2 a.m.

What we leave with clients

During ETL reliability assessments, we often spend the first deep-dive simply listing which green jobs have no reconciliation story. That list alone changes how teams talk about “successful” nights.