Field notes
When to split a monolith ETL job
Monolith extract-transform-load jobs often start as a convenience: one schedule, one credential store, one place to look when numbers look wrong. Over time they accumulate unrelated tables, and a single failure blocks unrelated reports.
Split when ownership diverges. If finance and product analytics both depend on the same job but escalate to different managers, separate schedules reduce political friction as much as technical risk.
Split when failure modes differ. A brittle third-party API should not share a transaction boundary with a stable internal ledger extract. Isolate retries and backoff so the stable path keeps landing.
Defer splits when the team cannot yet observe lineage. Cutting a job without documenting consumers creates silent gaps. Sketch dependencies first, then carve along those lines.
A practical test used in our assessments: if two parts of a job never share columns and never share incident tickets, they are candidates for separation in the next planning cycle.