- Experience
- 3+ yrs
- Salary
- USD 37 / hour
- Openings
- 1
- Posted
- 1 hour ago
- Work mode
- Work from home
- Education
- Bachelor's or Master's degree in relevant technical field preferred
- Resume
- Required to apply
Job description
About Mindrift and the Role
Mindrift is seeking experienced Python Data Scraping Engineers to contribute to the Tendem project, where human expertise and AI collaborate to enhance data extraction workflows. In this role, referred to as AI Pilot, you will work alongside automated Tendem Agents, providing key domain knowledge, critical thinking, and quality assurance to deliver accurate datasets.
Responsibilities
- Manage complete data extraction workflows from intricate websites ensuring comprehensive and precise dataset delivery.
- Utilize internal platforms like Apify and OpenRouter combined with custom-built solutions to optimize data gathering, verification, and execution processes.
- Adapt scraping methods to effectively extract data from dynamic and interactive web pages including JavaScript-rendered content.
- Maintain data integrity by implementing validations, cross-referencing sources, and adhering to formatting standards before submission.
- Scale operations efficiently by batching or parallel processing large volumes of data, monitoring errors, and ensuring resilience against minor website changes.
Candidate Requirements
- Minimum of three years’ experience in web scraping, data engineering, automation, or software development.
- A Bachelor’s or Master’s degree in Engineering, Applied Mathematics, Computer Science, or a related field is advantageous.
- Proficient in Python-based scraping tools such as BeautifulSoup and Selenium, with expertise handling dynamic content (JavaScript, AJAX, infinite scrolling) and API access through proxies.
- Demonstrated skill in extracting structured data from complex or inconsistent web page structures.
- Strong knowledge of data cleaning, normalization, validation, and delivering formats like CSV, JSON, or Google Sheets.
- Practical experience utilizing large language models (LLMs) and AI frameworks to improve automation workflows.
- Detail-oriented with a strong commitment to data accuracy.
- Ability to work autonomously and resolve technical issues independently.
- Providing a GitHub portfolio is a plus.
- English proficiency at upper-intermediate (B2) level or above is required.
Compensation
This freelance role offers earnings up to $37 per hour based on contribution level and speed. Rates may vary for different projects depending on their requirements.
Additional Information
- Fully remote work enables flexible scheduling using just a laptop and stable internet connection.
- Opportunity to gain valuable experience in a unique hybrid environment where AI and human collaboration drives innovation.
- Availability of performance-based bonuses rewarding quality and consistency.