Cloud Sr Technical Lead – D/A

Experience – 10+ Years

Job Description :

• 10+ years of overall experience in Data Engineering, Big Data, or Cloud Data Platforms.
• 5+ years of hands-on experience with Databricks (PySpark/Scala, Delta Lake, Lakehouse architecture).
• 4+ years of experience with AWS Glue (Glue ETL Jobs, Workflows, Crawlers, Glue Data Catalog).
• Experience with Azure ADLS Gen2 including RBAC, ACLs, and hierarchical namespace.
  Note: Candidates with AWS S3 experience can also be considered and trained on ADLS Gen2.
• 3+ years of experience designing and developing data ingestion and transformation pipelines using Spark-based frameworks.
• Basic understanding of IAM roles, network security, and cloud infrastructure concepts.
• Experience with Terraform or CloudFormation for infrastructure automation.
• Exposure to CI/CD tools such as Azure DevOps or GitHub Actions.
• Knowledge of data governance, metadata management, and performance optimization practices
• Lead the design and implementation of scalable and high-performing data engineering solutions.
• Architect and implement data ingestion, transformation, and storage solutions using Lakehouse architecture (Delta Lake, Medallion model).
• Develop and optimize Spark-based ETL/ELT pipelines using Databricks and AWS Glue.
• Integrate data from multiple sources into ADLS Gen2 or AWS S3 while ensuring proper security and access control.
• Establish coding standards, best practices, and conduct code reviews to maintain quality and consistency.
• Collaborate with cloud architects, DevOps teams, and stakeholders to ensure smooth deployment and operational readiness.
• Ensure data quality, governance, and lineage across the data platform.
• Monitor cloud resource usage and optimize performance and cost.

key skills-Databricks, AWS, S3, Adlsgen2, Azure