Head of Data Engineering

Position: Head of Data Engineering

Overview:
As the Head of Data Engineering for a scale up Digital Lender, you will play a critical role in designing, developing, and managing the data infrastructure that powers the digital platform. This position requires strong technical expertise in data engineering, AWS services, and financial data processing. You will lead a team of data engineers to ensure the efficient extraction, transformation, and loading of financial data into their analytics and decision-making systems.

Responsibilities:

Architectural Design: Design and architect scalable, efficient, and reliable ETL pipelines on AWS, ensuring high performance and low latency data processing.

AWS Service Utilisation: Utilise various AWS services such as S3, Glue, EMR, Athena, Redshift, and Lambda to build end-to-end data pipelines that meet business requirements and compliance standards.

Team Leadership: Lead and mentor a team of data engineers, fostering a culture of collaboration, innovation, and excellence. Provide guidance on best practices, code reviews, and technical troubleshooting.

Data Processing: Oversee the development and optimisation of data processing workflows, ensuring data integrity, accuracy, and completeness throughout the pipeline.

Performance Optimisation: Continuously optimise ETL processes for performance, scalability, and cost-efficiency. Identify bottlenecks and implement solutions to enhance pipeline throughput and reduce processing times.

Monitoring and Maintenance: Implement robust monitoring and alerting systems to proactively identify and address issues in the data pipeline. Perform routine maintenance and upgrades to ensure system reliability and stability.

Security and Compliance: Work closely with the security and compliance teams to implement data security best practices, encryption mechanisms, and access controls. Ensure compliance with relevant regulations such as GDPR, PCI-DSS, and SOC 2.

Collaboration: Collaborate cross-functionally with product managers, data scientists, software engineers, and other stakeholders to understand data requirements and deliver solutions that meet business needs.

Documentation: Create comprehensive documentation for ETL processes, data schemas, and system configurations. Ensure documentation is up-to-date and accessible to relevant stakeholders.

Stay Updated: Stay abreast of the latest advancements in data engineering, AWS technologies, and fintech industry trends. Evaluate new tools and technologies to improve the efficiency and effectiveness of the data pipeline.

Qualifications:

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
• Proven experience (5 years) in designing, building, and managing large-scale data pipelines on AWS.
• Strong expertise in AWS services such as S3, Glue, EMR, Athena, Redshift, and Lambda.
• Proficiency in programming languages such as Python, Scala, or Java for data processing and automation.
• Solid understanding of data modeling, data warehousing concepts, and SQL proficiency.
• Experience with financial data processing, including knowledge of financial instruments, market data, and regulatory requirements.
• Excellent leadership and communication skills, with the ability to effectively lead and inspire a team.
• Strong problem-solving skills and attention to detail, with a focus on delivering high-quality solutions.
• AWS certification (e.g., AWS Certified Big Data - Specialty) is a plus.

Join in revolutionising the fintech digital industry with cutting-edge data engineering solutions on AWS. Apply now to be part of the dynamic and innovative team!

InterQuest Group is acting as an employment agency for this vacancy. InterQuest Group is an equal opportunities employer and we welcome applications from all suitably qualified persons regardless of age, disability, gender, religion/belief, race, marriage, civil partnership, pregnancy, maternity, sex or sexual orientation. Please make us aware if you require any reasonable adjustments throughout the recruitment process.