Operations and QA Lead (AI Data Focus)
Oversaw data annotation quality assurance processes and RLHF workflow development for AI-powered publishing systems. Managed ground truth verification and developed SOPs for dataset annotation, ensuring data integrity and inter-annotator agreement. Automated large-scale data validation, reconciliation, and implemented structured guidelines to maintain quality benchmarks across distributed annotation teams. • Performed rigorous data reconciliation between source datasets and AI outputs using Python and Excel. • Designed and implemented Human-in-the-Loop (HITL) workflows to identify inconsistencies in AI-generated content. • Developed annotation guidelines and onboarded new labeling staff, improving consistency and scalability. • Collaborated with engineering to report data discrepancies and refine feedback loops.