Senior Big Data Engineers

• Design and evolve the overall data architecture, ensuring scalability, flexibility, and compliance with enterprise standards.
• Build efficient, secure, and reliable data pipelines using the Bronze-Silver-Gold architecture within EDL.
• Develop and orchestrate scheduled jobs in the EDL environment to support continuous ingestion and transformation.
• Implement Apache Iceberg for data versioning, governance, and optimization.
• Leverage the Medallion framework to standardize data product maturity and delivery.
• Govern metadata, data lineage, and business glossary using tools like Apache Atlas.
• Ensure data security, privacy, and regulatory compliance across all data processes.
• Support Data Mesh principles by collaborating with domain teams to design and implement reusable Data Products.
• Integrate data across structured, semi-structured, and unstructured sources from enterprise systems such as ODS and CRM systems.
• Drive adoption of DataOps/MLOps best practices and mentor peers across units.
• Generate and manage large-scale batch files using Spark and Hive for high-volume data processing.
• Design and implement document-based data models and transform relational models into NoSQL document-oriented structures (eg NoSQL Database or similar system).

• Bachelor’s, Master’s, or PhD in Computer Science, Data Engineering, or a related discipline.
• 5–7 years of experience in data engineering and distributed data systems.
• Strong hands-on experience with Apache Hive, HBase, Kafka, Solr, Elasticsearch.
• Proficient in data architecture, data modelling, and pipeline scheduling/orchestration.
• Operational experience with Data Mesh, Data Product development, and hybrid cloud data platforms.
• Familiarity with CRM systems, including CRM system, and data sourcing/mapping strategies.
• Proficient in managing metadata, glossary, and lineage tools like Apache Atlas.
• Proven experience in generating large-scale batch files using Spark and Hive.
• Strong understanding of document-based data models and the transformation of relational schemas into document-oriented structures.

• Expertise in data administration, modelling, mapping, collection, and distribution.
• Strong understanding of business workflows to support metadata governance.
• Hands-on experience with analytics and DWH tools (e.g., SAS, Oracle, MS SQL, Python, R Programming).
• Familiarity with data modelling tools (e.g., ERWIN), and enterprise databases (Oracle, IBM DB2, MS SQL, Hadoop, Object Store).
• Experience working across hybrid cloud environments (e.g., AWS, Azure Data Factory).
• In-depth knowledge of ETL/ELT processes and automation frameworks.
• Analytical thinker with strong problem-solving and communication skills.
• Able to collaborate effectively across technical and business teams.
• Proven ability to deliver high-quality outcomes within

Rapsys Technologies