Data Platform Engineer (Azure)

United States·Alexandriamid

Data EngineerData

0 views0 saves0 applied

Apply Now

Quick Summary

Key Responsibilities

Virtual Networks (VNETs) Private Endpoints Network Security Groups Access Security Policies CI/CD & Automation Build and maintain CI/CD pipelines using Azure DevOps and/or GitHub Actions.

Requirements Summary

Azure Monitor Log Analytics Alerting Tools Operational Dashboards Monitor platform performance and reliability.

Technical Tools

Data EngineerData

The Data Platform Engineer (Azure) is responsible for building, automating, and operating the infrastructure that supports the organization's enterprise data platform. This role ensures data pipelines, cloud environments, and platform services are reliable, scalable, secure, and observable across Microsoft Azure.

The position focuses on Infrastructure as Code (IaC), DevOps, CI/CD automation, cloud operations, platform governance, FinOps, security, and Site Reliability Engineering (SRE) practices. The Data Platform Engineer partners closely with Data Engineering, IT Infrastructure, Cybersecurity, and Business stakeholders to deliver a stable and high-performing data platform that supports analytics, reporting, and enterprise data initiatives.

Responsibilities

~1 min read

Design, implement, and maintain Infrastructure as Code (IaC) for Azure-based data environments using Terraform, Bicep, and/or ARM Templates.
Build and support highly available, scalable, and secure Azure cloud environments.
Configure and maintain Azure networking components including:
- Virtual Networks (VNETs)
- Private Endpoints
- Network Security Groups
- Access Security Policies

Build and maintain CI/CD pipelines using Azure DevOps and/or GitHub Actions.
Automate deployment of:
- Azure Data Factory
- Databricks Notebooks
- Data Pipelines
- Infrastructure Components
- Platform Services
Establish deployment standards and environment promotion processes across Development, QA, and Production environments.
Manage source control, versioning, branching strategies, and release processes.

Administer and govern Databricks workspaces.
Manage:
- Cluster Provisioning
- Cluster Policies
- Cost Controls
- Unity Catalog
- Access Controls
- Identity Management
Support data engineering teams by maintaining a reliable Databricks platform environment.

Manage Azure Data Lake Storage Gen2 environments.
Configure:
- Access Control Lists (ACLs)
- Lifecycle Management Policies
- Encryption Standards
- Storage Optimization Policies
Ensure compliance with data governance and security requirements.

Implement observability and monitoring strategies using:
- Azure Monitor
- Log Analytics
- Alerting Tools
- Operational Dashboards
Monitor platform performance and reliability.
Troubleshoot:
- Infrastructure Failures
- Performance Bottlenecks
- Platform Incidents
- Cloud Service Issues
Participate in incident response and root cause analysis activities.

Automate secrets and credential management using Azure Key Vault.
Support identity and access management initiatives.
Partner with Information Security teams to maintain compliance with governance and security standards.
Implement secure deployment and infrastructure management practices.

Monitor Azure and Databricks cloud consumption costs.
Analyze usage trends and identify optimization opportunities.
Recommend cost-saving initiatives while maintaining platform performance and reliability.

Document:
- Infrastructure Architecture
- Operational Procedures
- Runbooks
- Incident Response Processes
- Platform Standards
Establish best practices for platform reliability and operational support.

Requirements

~1 min read

Bachelor's degree in Computer Science, Information Systems, Engineering, or related field required.

5+ years of experience in:
- DevOps
- Platform Engineering
- Site Reliability Engineering (SRE)
- Cloud Infrastructure
- Related technical disciplines
3-5+ years of hands-on experience with:
- Microsoft Azure
- Databricks
- Azure Data Services
- Cloud Infrastructure Management