Data Annotation Specialist, Atlas Capture
As a Data Annotation Specialist at Atlas Capture, I ranked and scored assistant responses to support reinforcement learning from human feedback (RLHF) for large language model fine-tuning. I authored comprehensive annotation guidelines and performed daily calibration sessions to ensure annotation consistency and quality. I also performed PII detection and redaction across a substantial document corpus to comply with privacy standards. • Ranked and evaluated over 180,000 assistant responses per quarter to improve LLM alignment. • Created and maintained a 60-page annotation guideline covering multiple dialogue labeling dimensions. • Reduced inter-annotator disagreement from 14% to 6% and achieved zero compliance incidents for GDPR/CCPA. • Mentored 8 junior annotators with structured onboarding and regular feedback.