AI Trainer & Response Evaluator
As an AI Trainer and Response Evaluator at Outlier AI / Scale AI, I evaluated thousands of language model outputs across multiple domains. My primary responsibility was to assess the accuracy, helpfulness, coherence, and safety of AI-generated responses, providing preference rankings and written justifications for model training and RLHF tasks. I also designed challenging prompts to test model reasoning and identified hallucinations and policy issues in outputs. • Performed head-to-head RLHF and preference ranking of LLM responses with extensive written feedback. • Identified hallucinations, factual errors, bias, and policy violations in generative model outputs. • Created adversarial prompts and test cases to evaluate model weaknesses and instruction-following. • Consistently achieved documented accuracy rates above 93% for safety and quality evaluations.