Our client is a leading financial services organization dedicated to providing innovative insurance and wealth management solutions. You will oversee the implementation of an enterprise-wide data analytics platform, specializing in Azure and Databricks to design, develop, and manage scalable cloud-based data solutions while optimizing system performance and ensuring effective use of related services.

Job Duties:

• Design and implement a comprehensive data strategy using Azure Data, Delta Lake, Azure Databricks, and Azure Data Factory for advanced data processing and analytics.
• Lead the development of data pipelines and big data solutions on Azure Databricks, optimizing cluster performance, cost management, and resource utilization.
• Design, develop, and maintain robust ETL/ELT processes to ensure seamless data integration across various sources using Azure Data Factory and Databricks.
• Enforce data governance best practices via Databricks Unity Catalog and Azure Purview to maintain compliance, data quality, and integrity.
• Implement security measures for data protection, including Azure Key Vault and role-based access controls, while ensuring compliance with data privacy regulations.

Requirements:

• University or college graduate in Computing, IT, or a related discipline.
• Minimum 5 years of relevant experience, including at least 3 years of hands-on experience with Databricks.
• In-depth knowledge of Azure Databricks, Apache Spark, PySpark, and Delta Lake.
• Familiarity with Databricks components like Workspace, Run-time, Clusters, DLT, and Unity Catalog.
• Strong understanding of ETL/ELT processes and enterprise data models (e.g., CDM, departmental data marts).
• Expertise in Azure Data Factory for building complex data pipelines from various sources.
• Proficiency in SQL, Python, and PySpark for optimizing data workflows and managing data lakes/warehouses.
• Familiar with Azure Purview, Azure Key Vault, and role-based access control for data governance and security.

Our client offers competitive compensation and benefits, as well as a dynamic and fast-paced work environment. If you are looking for a challenging and rewarding career opportunity, apply to this exciting position by sharing your resume (in word format) with Twinkle yip ([email protected]) or call via 3972 5840.

For a comprehensive list of current opportunities, please visit www.connectedgroup.com