Migrating 80+ Production Jobs Off Legacy Compute Without Breaking a Single Pipeline

The Challenge

A fast-growing analytics team had accumulated dozens of production Databricks jobs on a legacy dedicated compute configuration that was becoming a cost and governance liability. No one had a reliable inventory of which jobs depended on legacy-only features, which made a platform-wide migration feel too risky to schedule.

The Approach

ELIX started with Discover: building a lightweight auditing script to classify every job by its actual dependencies — language runtime, init scripts, and job-creation method — rather than relying on tribal knowledge. The audit went through multiple iterations as edge cases surfaced. Once the real dependency map was clear, ELIX sequenced the migration in safe batches, validating each before moving to the next, and documented the findings so the team could self-serve future audits.

The Outcome

• Identified that a feared cross-job blocker (a shared init script) was already dead configuration — removing a false barrier that had stalled migration planning.
• Migrated the large majority of jobs to modern compute with zero production incidents.
• Left the team with a reusable audit tool instead of a one-time deliverable.

Tools & Stack