Field notes
Notes from data engineering engagements
Practical writing on pipelines, freshness, lineage, and migration checks — rooted in consulting work rather than generic productivity advice.
When to split a monolith ETL job
Not every long-running extract needs a rewrite. Here is how UK teams decide whether to carve a job by domain, schedule, or failure boundary.
Freshness SLAs that analysts actually trust
A warehouse can be 'up' while still late for the morning board pack. Freshness targets need calendars, not just uptime percentages.
Lineage notes that survive staff handovers
When a data engineer leaves, the next person inherits tribal knowledge. Lightweight lineage notes beat another abandoned wiki.
Dual-run checks without freezing reporting
Migrating a warehouse model does not require locking analysts out for a quarter. Dual-run patterns keep decisions moving.