Описание работы

About the Role

Rex.zone invites applications for a Senior Data Annotator to enhance AI and ML model training through meticulous data labeling and review efforts. This role is crucial in delivering top-quality, guideline-compliant annotated datasets for natural language processing, computer vision, and content safety applications.

Key Responsibilities

Lead advanced-level annotation and review processes across text, image, and multimodal data collections.
Conduct Reinforcement Learning from Human Feedback (RLHF) preference ranking and evaluate large language models following strict rubrics.
Execute quality assurance audits, calibration exercises, and continuous monitoring to uphold annotation standards.
Detect and report gaps in guidelines, catalog complex edge cases, and suggest improvements to rubric clarity.
Monitor consistency among annotators and resolve discrepancies via calibration protocols.
Participate in prompt evaluations assessing helpfulness, safety, and adherence to policies.
Engage in content safety labeling and the management of sensitive topic classifications.
Collaborate with engineering and data operations teams to enhance annotation tools, optimize workflows, and improve error identification.

Required Qualifications

Proven experience in data annotation, labeling, or quality assurance for machine learning datasets.
Aptitude to rigorously apply detailed rubrics and maintain compliance with annotation guidelines.
Understanding of RLHF processes, preference data appraisal, and large language model evaluation techniques.
Strong written communication skills necessary for documenting judgments and complex cases.
Hands-on experience with NLP tasks such as named entity recognition and/or computer vision annotation methods like bounding boxes and segmentation.

Quality Metrics

Performance is evaluated through regular audits, inter-annotator agreement assessments, rubric adherence, calibration rounds, and error rate analysis to ensure the highest integrity of training data for downstream models.

Remuneration

The position offers a competitive hourly wage ranging from $30 to $50 USD.

Senior Data Annotator