Principal Data Engineer-PySpark

This role requires a professional with a deep background in data engineering and expertise in designing and implementing cloud-based data warehousing solutions using Azure Synapse. The ideal candidate will be a strategic thinker with a proven record of accomplishment of delivering scalable and efficient data solutions to meet complex business requirements.

Responsibilities:

• Design and implement highly scalable, high performance data warehousing solutions in Azure Synapse.
• Analyze and migrate existing data systems to Azure Cloud, optimizing data flow and collection to improve data accuracy and utility.
• Contribute to data architecture design and modeling initiatives, ensuring alignment with business needs and compliance with data governance standards.
• Consult with stakeholders to define data requirements, deliverables, and timelines, providing technical leadership and guidance throughout project lifecycles.
• Implement data security and privacy policies in line with industry best practices and organizational standards.
• Optimize data integration pipelines for performance and scalability.
• Develop and maintain documentation regarding data architectures, procedures, and solutions.

Qualifications

• Bachelor’s or master's degree in computer science, Engineering, Information Technology, or a related field.
• 7+ years of experience in data engineering with a strong focus on cloud-based solutions.
• Expertise in Azure Synapse Analytics, including data integration, management, and security.
• Proficient with SQL, Python, PySpark, and other scripting languages commonly used in data engineering.
• Familiar with Azure IaC (Infrastructure as Code) and CI/CD automation using ARM, Bicep or Terraform
• Deep knowledge of ELT (Extract, Load, Transform) and ETL (Extract, Transform, Load) processes and experience with Azure Data Factory.
• Strong understanding of data modeling (Data Vault 2.0 experience or certification preferred), data warehousing, and data lakes.
• Proven consulting or advisory roles in the design and deployment of large-scale data solutions in Azure environments.
• Experience working directly with clients or stakeholders in a consulting capacity, translating business needs into technical specifications and actionable plans.
• Demonstrated ability to contribute to project leadership, coordinate across client functional groups, and manage client relationships.
• Microsoft Certified: Azure Data Engineer Associate preferrable 
• Microsoft Certified: Azure Solutions Architect Expert preferrable 
• Certified Data Vault 2.0 Practitioner (CDVP2) preferrable 

We have an amazing team of 700+ individuals working on highly innovative enterprise projects & products. Our customer base includes Fortune 100 retail and CPG companies, leading store chains, fast-growth fintech, and multiple Silicon Valley startups.

What makes Confiz stand out is our focus on processes and culture. Confiz is  ISO 9001:2015 (QMS),  ISO 27001:2022 (ISMS),  ISO 20000-1:2018 (ITSM) and  ISO 14001:2015 (EMS) Certified. We have a vibrant culture of learning via collaboration and making workplace fun.

People who work with us work with cutting-edge technologies while contributing success to the company as well as to themselves. 

To know more about Confiz Limited, visit: