LLM AI Quality Analyst (Personalization) - English

Role overview

This is a short-term remote contract for an LLM AI Quality Analyst focused on personalization. The assignment runs for one month, requires 20 hours per week, and begins immediately. Compensation is set at $11.5 per hour.

What you'll do

• Create multi-turn dialogue prompts that use personal context.
• Review personalized AI outputs for usefulness, relevance, and whether they are grounded in the provided context.
• Check whether the model uses personal information correctly or incorrectly.
• Compare two responses side by side and rank them based on quality.
• Spot grounding mistakes, weak assumptions, and overdone personalization.
• Prepare concise, well-organized explanations that refer to specific turns in the conversation.
• Pull out and validate model debug details and evidence of data-source usage.
• Keep evaluation data clean by removing conversation records after review.

Requirements

• Strong command of English, both written and reading.
• Background in data annotation, AI evaluation, content moderation, or a similar field.
• Sharp analytical ability and a strong eye for detail.
• Comfort evaluating subtle, unclear, or ambiguous AI responses.
• Willingness to use a primary personal Google account with data sources enabled.
• BS/BA degree, or equivalent relevant analytical experience.
• Clear written communication and the ability to give structured feedback.
• Ability to work independently in a remote environment.
• Access to a dependable desktop or laptop with stable internet.

Application process

• Submit the application form.
• Complete the ICF.
• Finish the assessment.

Engagement details

This is a short-term contract role based remotely, with a one-month engagement and an immediate start. The work commitment is 20 hours per week.