Senior Azure Databricks Engineer (ID:3774)

As a Senior Azure Databricks Engineer, you will: Design, develop, and maintain scalable and reliable data processing solutions using Azure Databricks. Build and manage robust streaming and batch data pipelines within complex Databricks environments. Design and optimize data models to support scalable processing and performance requirements. Manage multiple parallel data processing workflows using shared source datasets. Develop and maintain data transformation logic using Python, PySpark, and SQL. Implement and manage CI/CD pipelines using Azure and YAML-based configurations. Utilize Infrastructure as Code (ARM/Bicep) for deployment and infrastructure automation. Collaborate with cross-functional teams to ensure high-quality, maintainable, and reliable solutions. Continuously improve system stability, performance, and engineering practices while minimizing technical debt. Contribute actively within Agile teams to deliver scalable and sustainable data solutions. What You Bring to the Table: 8–10 years of overall professional experience, with strong expertise in Azure Databricks and Data Engineering. Deep hands-on experience with Azure Databricks as the primary technology platform. Strong proficiency in Python, PySpark, and SQL. Proven experience building and managing streaming and batch data processing solutions. Strong experience designing and maintaining scalable Databricks data models and architectures. Solid understanding of Azure cloud services and YAML-based CI/CD pipelines. Hands-on experience with Infrastructure as Code (ARM/Bicep). Experience working within Agile methodologies and delivery environments. Broader understanding of modern end-to-end data solution architectures is an advantage. Strong communication and collaboration skills. You Should Possess the Ability to: Build scalable, high-performance, and reliable data processing solutions. Design pragmatic and maintainable architectures while avoiding unnecessary complexity. Manage multiple data workloads and parallel processing scenarios efficiently. Troubleshoot, optimize, and improve complex data environments. Make well-informed technical decisions in evolving architectural landscapes. Collaborate effectively across engineering and business teams. Drive continuous improvement with a strong quality-first mindset. What We Bring to the Table: Opportunity to work on complex enterprise-scale cloud and data engineering initiatives. Exposure to modern Azure and Databricks-based data platforms. A collaborative Agile environment focused on innovation and engineering excellence. Opportunities for continuous learning, technical growth, and exposure to evolving technologies. A culture focused on quality, ownership, and long-term sustainable solutions. Let’s Connect Want to discuss this opportunity in more detail? Feel free to reach out. Recruiter: Hema Murali Phone: +31 20 369 0609 ; Extn :148 Email: hema.m@stafide.nl LinkedIn: https://www.linkedin.com/in/hema-murali-315999329/