Big Data Engineer

Posted 6 November by Randstad Sourceright
Easy Apply
Senior Data Engineer
3 month contract
£500 - £650 per day
Senior Data Engineer required to help my amazing digital client transform their data infrastructure to help create better insights and develop more intelligent products. The work you do will directly affect their ability to implement machine learning and AI into the business
As the Senior Data Engineer you will
  • Implement/develop pipelines to ingest data into a Snowflake data warehouse from a variety of data sources
  • Implement data transformation pipelines for real-time and batch environments
  • Support all product teams in adopting the data engineering tech stack to generate new data streams
  • Collaborate with Data Science and Business Intelligence teams to identify requirements and develop the necessary data workflows to deliver against those requirements
  • Ensure the data pipelines and data storage adheres to the information security standards
Who are you?
  • A Data Engineer with deep understanding of data technologies with a strong software engineering background, along with a deep interest in data analytics, machine learning and AI/ML.
  • Be able to create data solutions to process data at scale, both in batch and real-time pipelines
  • Solid experience in either Python and/or Java 8+ (Scala is a plus).
  • SQL expertise.
  • Relational and non-relational databases (Experience with Snowflake data warehouse is a plus)
  • Batch processing frameworks, such as Apache Airflow, DBT, etc.
  • Message brokers / stream processing technologies (Kinesis, Kafka, Storm, Spark Streaming, Flink, Beam etc.)
  • Architecting data pipelines
  • Familiarity and interest in cloud native technologies which includes AWS, Docker, Kubernetes, Amazon EKS, Terraform
  • Familiarity working with structured and unstructured data
  • Familiarity with continuous integration and deployment of code. Knowledge of Jenkins is a plus.
  • Experience working in an agile Test-Driven Development (TDD) environment and extreme programming (XP)

This is an amazing opportunity to work with a really cool company undergoing a transformation to be AI ready, if you’re interested in hearing more I’d love to hear from you

Reference: 39310346

Bank or payment details should never be provided when applying for a job. For information on how to stay safe in your job search, visit SAFERjobs.

Report this job