Senior Software Engineer, Data Engineering

Design, develop, and maintain data ingestion, transformation, and orchestration pipelines (batch and real-time). 

Build and optimize data Lakehouse architectures using Azure Synapse, Delta Lake, or similar frameworks. 

Integrate and manage structured and unstructured data sources (SQL/NoSQL, files, documents, IoT streams). 

Develop and operationalize ETL/ELT pipelines using Azure Data Factory, Databricks, or Apache Spark. 

Collaborate with Data Scientists to prepare and serve ML-ready datasets for model training and inference. 

Implement data quality, lineage, and governance frameworks across pipelines and storage layers. 

Work with BI tools (Power BI, Superset, Tableau) to enable self-service analytics for business teams. 

Deploy and maintain data APIs and ML models in production using Azure ML, Kubernetes, and CI/CD pipelines. 

Ensure scalability, performance, and observability of data workflows through effective monitoring and automation. 

Collaborate cross-functionally with engineering, product, and business teams to translate insights into action. 

Experience: 4 to 8 years in Data Engineering or equivalent roles. 

Programming: Strong in Python, SQL, and at least one compiled language (C#, Java, or Scala). 

Databases: Experience with relational (SQL Server, PostgreSQL, MySQL) and NoSQL (MongoDB, Cosmos DB) systems. 

Data Platforms: Hands-on experience with Azure Data Lake, Databricks. 

ETL/ELT Tools: Azure Data Factory, Apache Airflow, or dbt. 

Messaging & Streaming: Kafka, Event Hubs, or Service Bus for real-time data processing. 

AI/ML Exposure: Familiarity with ML frameworks (TensorFlow, PyTorch) and MLOps concepts. 

Visualization & Analytics: Power BI, Apache Superset, or Tableau. 

Cloud & DevOps: Azure, Docker, Kubernetes, GitHub Actions/Azure DevOps. 

Best Practices: Solid understanding of data modeling, version control, and CI/CD for data systems. 

Experience in knowledge graph or semantic search solutions. 

Understanding of LLM-based data retrieval (RAG) patterns. 

Exposure to data mesh, data fabric, or domain-oriented data architecture. 

Familiarity with MLflow, Delta Live Tables, or DataBricks Unity Catalog.