Technical Lead Data Engineer (Databricks)

Job Description

As a Technical Lead, you will be responsible for technical leadership of Agile Team(s) within our Business Analytics & Insights (BAI) team. You will work closely with cross-functional teams to deliver high-quality solutions in domains such as Supply Chain, Finance, Operations, Customer Experience, HR, Risk Management, and Global IT.

• Develop a comprehensive technical plan for migration, including data ingestion, transformation, storage, and access control in Azure's Data Factory and data lake.
• Design and implement scalable and efficient data pipelines to ensure smooth data movement from multiple sources using Azure Databricks.
• Developing scalable and re-usable frameworks for ingesting data sets.
• Ensure data quality and integrity throughout the entire data pipeline, implementing robust data validation and cleansing mechanisms.
• Working with event based / streaming technologies to ingest and process data.
• Provide technical guidance and support to the team, resolving any technical challenges or issues that may arise during the migration and post-migration phases.
• Stay up to date with the latest advancements in cloud computing, data engineering, and analytics technologies, and recommend best practices and industry standards for implementing the data lake solution.

Qualifications

Technical skills :

Soft skills :

Additional Information

Nice-to-have skills / certifications :