Links is working with a global e-commerce company based in London who are looking for a perm Principal Big Data Engineer. This is someone who is an experienced senior technologist with a depth of experience in leading multi-discipline teams through the full delivery lifecycle of complex data products and pipelines with a clear understanding of at-scale data processing solutions.
This is an incredibly senior role within a global company and therefore you need to have significant team lead experience (with professionals such as Data Scientists, DevOps Engineers & Product Manager) as well as used to being the technical "goto" person. As well as this you will have experience in both engineering and architecture across technology discipline and have a successful background working in a fast paced, data driven company - ideally in a high transactional e-commerce environment.
Most of these technical skills are needed:
- Proven knowledge of the Data Science Lifecycle
- Demonstrable experience of these software engineering languages Java 8, Scala 10/11, Maven / SBT
- Python (preferably with pandas, numpy and/or sci-kit learn)
- C# and preferably some F#
- Great understanding of the processes and principles of machine learning systems
- Good knowledge of HPC/Batch computing and GPU offload scenarios for MPI/parametric sweep
- Strong knowledge of relational databases and strong associated knowledge of TSQL, plus NoSQL document and time series databases including CosmosDB, MongoDB, Cassandra
- Knowledge of data orchestration technologies such as Oozie, AirFlow, Apache NiFi or Azure Data Factory
- Sound knowledge of the Apache Big Data stacks such as Hadoop and Spark including, HDFSMapReduce, YARN
- Good understanding of real-time streaming technologies such as Apache Kafka, Azure EventHub, Spark Streaming, Apache Storm, Apache Flinketc
- Strong knowledge of Microsoft Azure data capabilities (Azure Data Factory, Azure Stream Analytics, SQL DB / DW, CosmosDB, Azure Data Lake etc)
- Good knowledge in columnar data formats such as Parquet, ORC
- Good data visualisation skills including proven usage of PowerBI or Tableau
- Good understanding about the container technologies such as Docker and Kubernetes
- Knowledge and proven usage of DevOps technology, preferably the standard Azure Resource Manager, Team City, Octopus and VSTS set that ASOS standardises on, but understanding of the approach and tools more important
- Great understanding of modern data architecture, service-oriented, API based and load levelling application design principles, lambda, streaming and micro-batch architectural knowledge and experience
If you are interested in hearing more about the client, please apply and I would be more than happy to have a chat!
- Distributed Systems
- Data Science
- Big Data