EFS FCRM 170694

Key Responsibilities: Design, build, and maintain robust ETL/ELT data pipelines using Apache Spark on Databricks. Implement data Lakehouse architecture using Delta Lake for cost-effective data storage and analytics. Use Databricks Workflows for orchestrating batch and streaming pipelines. Develop and maintain CI/CD pipelines for data applications using tools such as AWS, GitHub Actions or Databricks Repos. Monitor pipeline performance and troubleshoot data issues in real-time and batch environments. Document solutions, workflows, and technical standards. Required Skills: Experience in data engineering with a strong focus on AWS Databricks and Apache Spark. Proficiency in PySpark, SQL, and Python. Experience with Delta Lake, Databricks SQL, and Unity Catalog. Hands-on experience with cloud platforms Familiarity with data lakehouse architecture, data warehousing and streaming data Strong understanding of ETL best practices, data partitioning, and performance tuning. Experience with CI/CD for data pipelines. Experience in Machine learning and Data Analytics is added advantage. Strong exposure and understanding on the AI and Agentic AI technologies. Excellent problem-solving and communication skills ##LI-DNIKey Responsibilities: Design, build, and maintain robust ETL/ELT data pipelines using Apache Spark on Databricks. Implement data Lakehouse architecture using Delta Lake for cost-effective data storage and analytics. Use Databricks Workflows for orchestrating batch and streaming pipelines. Develop and maintain CI/CD pipelines for data applications using tools such as AWS, GitHub Actions or Databricks Repos. Monitor pipeline performance and troubleshoot data issues in real-time and batch environments. Document solutions, workflows, and technical standards. Required Skills: Experience in data engineering with a strong focus on AWS Databricks and Apache Spark. Proficiency in PySpark, SQL, and Python. Experience with Delta Lake, Databricks SQL, and Unity Catalog. Hands-on experience with cloud platforms Familiarity with data lakehouse architecture, data warehousing and streaming data Strong understanding of ETL best practices, data partitioning, and performance tuning. Experience with CI/CD for data pipelines. Experience in Machine learning and Data Analytics is added advantage. Strong exposure and understanding on the AI and Agentic AI technologies. Excellent problem-solving and communication skills