Senior Data Engineer

Data is coming out of DB2 and a long list of other source systems, landing in Snowflake through Fivetran, and getting shaped by dbt into something the business can trust. The pieces are in place. What's missing is the senior engineer who sets the standard for how it all gets built, deployed, and run, so pipelines stop behaving like one-off projects and start behaving like a platform.

You'll own the design and governance of the ELT pipelines that feed analytics and reporting across a large financial services organization. This is a hands-on technical lead seat. You write the code, set the patterns others follow, review architecture, and mentor the engineers around you. Everything runs on AWS and infrastructure lives in Terraform. When a pipeline breaks, a Snowflake bill spikes, or a new tool needs a yes or no, you're the one people look to.

Build and tune Fivetran ingestion and CDC out of DB2 and other sources, handle schema changes without breaking downstream models, and hold the line on ingestion SLAs. Structure dbt projects into staging, intermediate, and mart layers, write the macros and tests that keep them consistent, and make lineage and documentation something people actually use.

Shape the Snowflake architecture: warehouse sizing, workload isolation, clustering, access control, secure sharing, and the cost conversation nobody else wants to have. Provision Snowflake objects and AWS infrastructure through Terraform so every environment is repeatable and auditable.

Own the CI/CD path for data assets, from linting and automated validation through dev, test, and prod promotion that compliance can trace. Write Python, PySpark, and SQL where dbt isn't the right tool, and set the standards the team writes to. Run proofs of concept on new tooling and make the call on what earns a spot in the stack. Work with security and compliance so controls hold without slowing the team to a crawl.

Someone who has already built this kind of platform and has opinions about what they'd do differently next time. You can explain why you sized a warehouse the way you did, why a model lives in intermediate instead of a mart, and what a bad schema change cost you once. You spot problems before anyone asks, and you can walk a business stakeholder through a technical tradeoff without losing them.

• 7 to 8+ years of hands-on data engineering, with real production time in a heavy AWS environment (required)
• Snowflake architecture and performance tuning at enterprise scale
• Fivetran connector configuration, CDC, and schema evolution
• Terraform for cloud infrastructure and Snowflake object management
• DB2 as a source system: you know how data comes out of it and where it bites
• dbt in production: modular models, macros, tests, documentation, version control
• CI/CD for data pipelines and a history of leading or mentoring engineers
• Bachelor's in a data-centric field or equivalent experience

Kafka or other streaming work, PySpark at scale, Azure exposure, and time in banking or another regulated industry.

Interested candidates can apply directly or contact doug.halve@techpartnersllc.com for more information