Mistral21mo ago
Web Crawling Engineer
Paris · Paris.Full-timemid
OtherWeb Crawling Engineer
3 views0 saves0 applied
Quick Summary
Key Responsibilities
• Developing and maintaining web crawlers using Go to extract data from target websites. • Utilize headless browsing techniques, such as Chrome DevTools,
Technical Tools
OtherWeb Crawling Engineer
At Mistral AI, we believe in the power of AI to simplify tasks, save time, and enhance learning and creativity. Our technology is designed to integrate seamlessly into daily working life.
We democratize AI through high-performance, optimized, open-source and cutting-edge models, products and solutions. Our comprehensive AI platform is designed to meet enterprise as well as personal needs. Our offerings include Le Chat, La Plateforme, Mistral Code and Mistral Compute - a suite that brings frontier intelligence to end-users.
We are a dynamic, collaborative team passionate about AI and its potential to transform society. Our diverse workforce thrives in competitive environments and is committed to driving innovation. Our teams are distributed between France, USA, UK, Germany and Singapore. We are creative, low-ego and team-spirited.
Join us to be part of a pioneering company shaping the future of AI. Together, we can make a meaningful impact. See more about our culture on https://mistral.ai/careers.
About Mistral
At Mistral AI, we believe in the power of AI to simplify tasks, save time, and enhance learning and creativity. Our technology is designed to integrate seamlessly into daily working life.
We democratize AI through high-performance, optimized, open-source, and cutting-edge models, products, and solutions. Our comprehensive AI platform is designed to meet enterprise needs, whether on-premises or in cloud environments. Our offerings include le Chat, the AI assistant for life and work.
We are a dynamic, collaborative team passionate about AI and its potential to transform society.
Our diverse workforce thrives in competitive environments and is committed to driving innovation. Our teams are distributed between France, the USA, the UK, Germany, and Singapore. We are creative, low-ego, and team-spirited.
Join us to be part of a pioneering company shaping the future of AI. Together, we can make a meaningful impact. See more about our culture on https://mistral.ai/careers.
Role Summary
We are looking for a skilled and motivated Web Crawling Engineer to join our dynamic engineering team. The ideal candidate should have a solid background in distributed web crawling, scraping and data extraction, with experience using advanced tools and technologies to collect and process large-scale data from diverse web sources at large scale.
What you will do
As a Web crawling engineer, you will be responsible for:
• Developing and maintaining web crawlers using Go to extract data from target websites.
• Utilize headless browsing techniques, such as Chrome DevTools, to automate and optimize data collection processes.
• Collaborate with cross-functional teams to identify, scrape, and integrate data from APIs and web pages to support business objectives.
• Create and implement efficient parsing patterns using tokenizers, regular expressions, XPaths, and CSS selectors to ensure accurate data extraction.
• Design and manage distributed job queues using technologies such as Redis, Aerospike and Kubernetes to handle large-scale distributed crawling and processing tasks.
• Develop strategies to monitor and ensure data quality, accuracy, and integrity throughout the crawling and indexing process.
• Continuously improve and optimize existing web crawling infrastructure to maximize efficiency and adapt to new challenges.
About you
Core programming and web technologies
• Proficiency in Go (Golang)/Rust/Zig for building scalable and efficient web crawlers.
• Deep understanding of TCP, UDP, TLS and HTTP/1.1,2,3 protocols and web communication.
• Knowledge of HTML, CSS, and JavaScript for parsing and navigating web content.
• Familiarity with cloud platforms (AWS, GCP), orchestration (Kubernetes, Nomad), and containerization (Docker) for deployment.
Data Structures & Algorithms
• Mastery of queues, stacks, hash maps, and other data structures for efficient data handling.
• Ability to design and optimize algorithms for large-scale web crawling.
Web Scraping & Data Acquisition
• Hands-on experience with networking and web scraping libraries.
• Understanding of how search engines work and best practices for web crawling optimization.
Databases & Data Storage
• Experience with SQL and/or NoSQL databases (knowing Aerospike is a bonus) for storing and managing crawled data.
• Familiarity with data warehousing and scalable storage solutions.
Distributed Systems & Big Data
• Knowledge of distributed systems (e.g., Hadoop, Spark) for processing large datasets.
Bonus Skills (Nice-to-Have)
• Experience with web archiving projects & tooling, open-source archiving is a big plus!
• Experience applying Machine Learning to improve crawling efficiency or accuracy.
• Experience with low-level networking programming and/or userspace TCP/IP stacks.
Hiring Process
Here is what you should expect:
• Introduction call - 35 min
• Hiring Manager Interview - 30 min
• Live-coding Interview - 45 min
• System Design Interview - 45 min
• Deep dive interview (optional) - 60min
• Culture-fit discussion - 30 min
• Reference checks
By applying, you agree to our Applicant Privacy Policy.
Listing Details
- Posted
- July 12, 2024
- First seen
- March 25, 2026
- Last seen
- April 21, 2026
Posting Health
- Days active
- 27
- Repost count
- 0
- Trust Level
- 25%
- Scored at
- April 22, 2026
Signal breakdown
freshnesssource trustcontent trustemployer trust
External application · ~5 min on Mistral's site
Please let Mistral know you found this job on Jobera.
4 other jobs at Mistral
View all →Explore open roles at Mistral.
Similar Web Crawling Engineer jobs
View all →
Browse Similar Jobs
Healthcare7.3kPersonal Trainer4.9kIn Home Personal Trainer4.8kFitness & Wellness2.5kSoftware Engineer Intern2.1kHuman Resources (legacy human-resources)1.8kBehavioral Health1.8kAmerican Logistics Authority1.8kPart1.6kSocial Work & Counseling1.5kManager1.4kPart Time Assistant Manager1.3kSocial Worker937Director851Staff Engineer848Adult Social Worker766Hospitality & Food Service749Consultant728Adult Social Worker Mash/Safeguarding709Social Work707
Newsletter
Stay ahead of the market
Get the latest job openings, salary trends, and hiring insights delivered to your inbox every week.
A
B
C
D
No spam. Unsubscribe at any time.
Web Crawling Engineer