Module Lead / Technical Lead – GCP Data Engineer

Location: Hyderabad or Bangalore

• Experience: 6 to 10 Years

Job/Role Description:

• At least 2+ years in GCP data engineering development in overall experience 
• Experience in developing data engineering, data migration process using scripts (Python, PySpark) with utilities like Rclone and services like Cloud Data Fusion, Dataproc, Dataflow
• Experience in working on storage services of Cloud Storage, Big Query
• Expertise in design and development of Data pipelines and ETL/ELT using ADF, Databricks to move data from relational/structured/unstructured data from source to data lake to data warehouse, on Azure.
• Creating Pipelines and integrations of multiple data sources using Azure data factory and other integration technologies.
• Good to have batch and streaming (Kafka, Spark Streaming, Stream Analytics.
• Good to have Big data processing and transformation using Databricks (Scala/Python)
• Good to have Expertise in optimizing cost and performance of data pipelines.
• Good to have experience to ingesting video files.
• Good understanding of Azure, and AWS
• Good understanding of ML on GCP
• Good communication and teamwork skills

Key Skills: GCP, Azure