Position Name : Data Engineer Job Summary: Responsible for the design, development and support the implementation and maintenance of cloud data platform pipelines and data models both globally and across different markets/countries. Job Description: Design, develop, document and implement end-to-end data pipelines and data integration processes, both batch and real-time. This includes data analysis, data profiling, data cleansing, data lineage, data mapping, data transformation, developing ETL / ELT jobs and workflows, and deployment of data solutions. Monitor, recommend, develop and implement ways to improve data quality including reliability, efficiency and cleanliness, and to optimize and fine-tune ETL / ELT processes. Recommend, execute and deliver best practices in data management and data lifecycle processes, including modular development of ETL / ELT processes, coding and configuration standards, error handling and notification standards, auditing standards, and data archival standards. Ensure development adhere to guidelines and governance policies on data and business intelligence platform. Collaborate with IT team members, SMEs, vendors and internal business stakeholders, to understand data needs, gather requirements and implement data solutions to deliver business goals. BAU support for any data issues and change requests, document all investigations, findings, recommendations and resolutions. Job Requirement: Hands-on experience on Azure Big Data Solution such as Azure Spark pool, Data Factory, Databricks, Gen2, Synapse, Power BI. Experience with various of ETL/ELT frameworks, data warehousing concepts, data management framework and data lifecycle processes. Experienced in handling and processing different types of data (structured, semi-structured and unstructured). Strong knowledge in various database technologies (RDBMS, NoSQL and columnar). Understanding of ETL programming languages like Python, Scala, Java, Spark, Shell and PLSQL. Ability to communicate and present technical information in a clear and unambiguous manner. Strong ability to work independently and cooperate with diverse teams in a multiple stakeholder's environment. Strong sense of work ownership, high affinity with anything data and a desire for constant improvements. Experience with SAP data sources preferred. Education Bachelor's degree in Computing/Information Technology or equivalent. Work Experience Up to three years' experience in similar role.
