Senior Python Software Engineer - Web Crawling

Type: Full-Time, Permanent
Location: London, Farringdon

Who are we

Pricesearcher is a fast-growing UK startup that has been going since December 2016. We’re a team of 20, 15 working in tech, situated in the heart of Farringdon, London. We’re a vertical search engine for product, like Skyscanner for flights or Indeed for jobs. Our mission is to index all the world’s products and prices and create an open and transparent shopping search experience for everyone. We have raised £4.3million in seed funding and we are aiming to raise series A in summer 2020 that will allow us to further grow and expand.


As an engineer at Pricesearcher you will get to work on a wide range of exciting and challenging problems with our data set of over a billion products. This particular opportunity specialises on our crawler products. You will be working on a mix of brown and green field projects that support the crawler ingestion channel for our product search engine. Our crawler is built using Python 3 execute on AWS Lambda across multiple regions and also uses SNS, SQS, Kinesis to fetch information from tens of thoushand of retailers across more than 20 countries. We are a fast moving team and with this role comes an excellent opportunity to very quickly make a meaningful contribution to the business.

Our tech stack includes

  • Python (APIs, Crawler and Data Ingestion and Processing Pipelines)
  • Spark/PySpark, Presto, Parquet
  • MySQL
  • Elasticsearch
  • AWS (Athena, EC2, CloudFront, Lambda, DynamoDB, SQS, SNS, Kinesis, S3 + more!)
  • Jenkins
  • Terraform + Packer
  • Docker + Docker Compose


  • Mentor and assist existing members of the team
  • Taking ownership over projects and leading the technical direction of them
  • Support the crawler ingestion channel for our product search engine by working on greenfield as well as existing projects
  • Collaborate with other members of the development team on existing and new projects


  • 5+ years of Python software development experience and its surrounding tooling and libraries - we are using 3.6 at the moment, but happy to consider 2.7 experience
  • Commercial experience with large scale web crawling or scraping
  • Experience with a variety of AWS services, specificially with at least one of: Lambda, SNS, SQS, Kinesis
  • Experience with serverless frameworks
  • Understanding of how to write maintainable and testable code
  • Understanding of version control systems such as Git
  • Ability to work in a fast paced environment
  • Self motivated and able to learn independently


  • Experience with scalable systems and architectures
  • Experience with Infrastructure as Code - Terraform / Ansible
  • Experience with Elasticsearch, MySQL, Redis
  • Experience with CI/CD, via Jenkins preferably
  • Experience working in an agile software development environment
  • Experience working in an early stage startup

What you’ll get in return

  • Generous company benefits: EMI Share Options, 25 days of holidays (+ bank holidays), Workplace pension, Weekly team lunches, Regular team social events
  • You will not be bored, you will work on hard and challenging real-world problems
  • An environment that encourages learning and experimentation
  • Ability to work on greenfield products that you can own outright
  • You will expand your knowledge on systems, architecture and data processing
  • Competitive salary and generous equity package
  • Great central London location and an AWESOME team!

Our challenges keep us awake at night and take our thinking to the next level. We approach our problems from all angles as a team and believe that failing is just as important to the process as succeeding. You will work directly with the founding team who have been together for over 5 years. People from all walks of life who have joined from some of the biggest names in online, to work together on this mission. Every member of the business is here to help support and share in your success!

If you think you’re the right candidate for this position, email us at