Field notes
Field notes
Practical notes on data pipeline engineering, ETL reliability, and operational habits from Web Willow Hub consultants.
Short essays from our consulting work — failure patterns, reconciliation habits, and decisions that show up repeatedly on Taiwanese pipeline teams.
When the job is green but the ledger is wrong
Exit codes lie. How reconciliation assertions catch silent ETL truncation before finance notices.
Retry storms and the polite API that hates you
How aggressive ETL retries amplify upstream outages — and how to back off without losing the night’s load.
Batch windows around the harbor clock
Scheduling ETL around Kaohsiung operations realities — shift changes, customs cutoffs, and why UTC-only thinking fails locally.
Lineage you can explain at the whiteboard
Lightweight lineage practices for ETL teams that cannot staff a full catalog program — and still need to answer ‘where did this number come …