AI Trainer / Data Annotator
As an AI Trainer and Data Annotator at Crowdgen by Appen, I evaluated over 50,000 LLM responses for accuracy, relevance, clarity, and safety using detailed rubrics. I performed ranking and comparison on AI prompt outputs and created custom evaluation rubrics for nuanced text such as historical facts and creative content. I ensured consistent guideline compliance and high-fidelity data for AI model training while managing high-volume annotation workflows. • Applied rubric-based scoring to text responses for LLM evaluation. • Flagged and documented inconsistencies to provide structured justifications. • Developed custom rubrics for specialized subject matter including historical and creative textual data. • Automated repetitive annotation tasks with Python scripting for increased efficiency.