Module Lead – Azure Data Engineer

Job Description:

• Experience – 4 – 8 Years
• Expertise in design and development of Data pipelines and ETL/ELT using ADF, Databricks to move data from relational/structured/unstructured data from source to data lake to data warehouse, on Azure.
• Creating Pipelines and integrations of multiple data sources using Azure data factory and other integration technologies
• Good to have batch and streaming (Kafka, Spark Streaming, Stream Analytics)
• Good to have big data processing and transformation using Databricks (Scala/Python)
• Good to have Expertise in optimizing cost and performance of data pipelines

Primary Skills:

• Expertise with minimum 4+ projects (7+ Yrs exp.)
• Databricks, Python
• Azure SQL DB/DW
• Experience in implementing effective database solutions and data models to store and retrieve data
• Designing DB schema, data models on RDBMS and DW
• Programming and optimizing DB objects Views, Stored Procs, Functions
• Azure Data Lake or Blob storage
• Experience working with high volume data, large objects.

Key Skills: Azure SQL, Databricks, Python, Data Lake or Blob storage