Data Engineer

Data Development

• Manage deployment of changes, upgrades and bug fixes to databases, data lake and data warehouse
• Development of new AWS Data Lake
• Migrate customer orchestration solution to AWS Airflow
• Refine and extend system and ETL designs to improve accuracy, timeliness, completeness, and consistency
• Manage integration and maintenance of new data sources. Design, build and maintain data pipelines
• Code management, review and reversion using GitHub

Data Administration

• Monitoring, diagnosis and troubleshooting orchestration including SSIS, Airflow, Spark, Scala, Kubernetes, AWS Orchestration, Python and Glue
• Security, key management, DR, performance, and cost monitoring of AWS solution including S3 and Athena
• Disaster Recovery planning, maintenance, and execution
• Design, develop and maintain non-production environments
• Design, develop and adhere to change control processes
• Administration of MSSQL Server RDS environment
• Upgrade of existing MSSQL Server environment including SSIS, SSAS, MDS
• Establish the needs of users, monitor user access and security, control access permissions and privileges
• Ensure that storage and archiving procedures are functioning correctly
• Develop, manage, and test back-up and recovery plans
• Manage the security and disaster recovery aspects of Databases and Data Lake

Data Governance

• Align existing and new processes to defined design patterns and coding standards
• Maintain data standards, including adherence to the Data Protection Act
• Investigation, design and implementation of QA processes and reporting
• Protection of Personally Identifiable Information
• Design and develop GDPR processes
• Collation and maintenance of meta data about the Hearst Data Estate.
• Create and maintain documentation, including data standards, procedures, and definitions for the data dictionary (metadata)
• Maximise data value to the business

Application Support

• Commission and install new applications and customise existing applications to make them fit for purpose
• SSRS, PowerBI, Tableau and Looker report editing and origination
• Data Warehouse (DW) maintenance and support. SSIS, SSAS, SQL Server for analysis on existing DW
• Design and development of replacement for existing MSSQL Data Warehouse and Cube
• Advice on back-end standards for new and existing applications and ETL
• Creation and maintenance of GDPR Compliant Development and Testing environments
• Design, implementation and monitoring of deployments along with co ordinating code, data and schema reversions when required.

Stakeholder Management

• Liaise with developers, application owners and SMEs around DR, Security and Operational Requirements (including ETL timings)
• Liaise with data architect to move data estate to consolidate, simplify and document operational landscape
• Define and enforce standards for data quality
• Define and enforce code standards
• Liaise with 3rd party data suppliers to understand data structures / formats etc.
• Monitor performance and manage parameters to provide fast responses to front-end users
• Manage and respond to alerts and notifications around feeds to our Databases and Data Lake
• Communicate regularly with technical, applications and operational staff to ensure database integrity and security

Team

• Positive contributor to the wider Data & Technology team
• Training and mentoring of junior team members
• Provide wider business user support by responding to system issues within agreed SLAs

Leading from the front

• To be an ambassador of Hearst and its values.
• You may be required to undertake such further duties as your skills, qualifications and experience allow and/or as may be assigned to you from time to time.