Position: Senior Data Engineer Location: Bangsar South, Malaysia Contract Duration: 12 months Role Purpose We are looking for a Senior Data Engineer to design, build, and maintain scalable data pipelines, ensuring data quality, reliability, and accessibility for analytics and reporting needs. Key Responsibilities Data Pipeline Development Collaborate with cross-functional teams to understand data requirements and design efficient data pipelines. Develop, implement, and maintain ETL processes to collect, clean, and store data from various sources. Optimize and enhance existing data pipelines for better performance, scalability, and reliability. Data Quality & Operations Ensure data quality and integrity through validation, monitoring, and error handling. Troubleshoot and resolve data-related issues in a timely manner. Collaboration & Support Partner with data scientists and analysts to provide necessary data sets and tools for advanced analytics and reporting. Implement data security and privacy best practices to protect sensitive information. Continuous Improvement Stay updated with the latest trends and technologies in data engineering. Contribute to the continuous improvement of data infrastructure. Technical Focus Must have hands-on experience with Databricks for building and optimizing data pipelines and performing data engineering tasks. Requirements Education Bachelor's degree in Computer Science, Information Technology, or related field (or equivalent work experience). Experience 5+ years of experience in data engineering or a related role. Technical Skills Proficiency in Python, Java, or Scala. Strong experience with data warehouse technologies (AWS Redshift, Google BigQuery, Snowflake). Hands-on experience with ETL tools and frameworks (Apache Spark, Apache NiFi). Experience with Qlik. Hands-on experience with Databricks. Knowledge of database systems (SQL and NoSQL) and data modeling. Familiarity with data integration and orchestration tools (Apache Airflow). Experience with version control systems (Git). Soft Skills Understanding of data security and compliance standards. Excellent problem-solving and communication skills. Ability to work independently and collaboratively in a fast-paced environment. Strong attention to detail and commitment to delivering high-quality work.
