Python Scraping Engineer

Senior Python Scraping Engineer - Up to £70,000

Location: UK (Remote)
Permanent

A well-established technology organisation is seeking a Senior Python Engineer to design, build and operate robust web data collection systems at scale. This role focuses on gathering structured data from complex, high-traffic websites where standard approaches are ineffective due to dynamic content and defensive measures.

You will work closely with engineering and data teams to ensure data pipelines remain stable, accurate and performant in fast-changing environments.

Responsibilities:

• Design and maintain scalable Python-based web data collection systems
• Develop solutions for dynamically rendered, JavaScript-driven websites
• Adapt to frequent structural and behavioural changes across target sites
• Implement monitoring and alerting to identify data quality or system issues early
• Optimise throughput, reliability and cost across data collection infrastructure
• Apply strategies to manage access restrictions, rate limits and traffic controls
• Troubleshoot and resolve production issues efficiently
• Collaborate cross-functionally and maintain clear technical documentation

Required Experience:

• Strong commercial experience with Python in production environments
• Proven background building and maintaining web data collection or extraction systems
• Solid understanding of web technologies, request flows and browser behaviour
• Hands-on experience with browser automation frameworks
• Experience working with concurrent or asynchronous processing
• Comfortable operating systems at scale in cloud-based environments
• Strong analytical and debugging skills

Why Apply:

• Work on technically challenging systems with real-world scale
• High level of autonomy and ownership
• Collaborative engineering culture
• Remote working
• Competitive salary and benefits package