AI Data Trainer & LLM Evaluation Lead
Led a distributed team of annotators to evaluate and fine-tune large language model outputs for safety, factual accuracy, and instruction adherence. Developed and maintained RLHF annotation guidelines and rubrics used in multiple LLM training projects. Oversaw advanced red-teaming and adversarial prompt testing to expose model failure modes and biases. • Curated and annotated a benchmark dataset of 5,000+ prompt-response pairs for creative and factual Q&A. • Delivered weekly quality reports with inter-annotator agreement analyses, achieving a 98% consistency score. • Designed multi-dimensional scoring rubrics for comprehensive LLM output evaluation. • Used internal/proprietary tools and collaborative platforms for annotation and reporting.