AI Data Annotator / Evaluator
As an AI Data Annotator and Evaluator at Outlier.ai, I performed in-depth transcription and linguistic annotation on spontaneous speech datasets. My tasks included rich transcription, disfluency tagging, prosodic markup, non-speech event labeling, and the evaluation of AI-generated responses for accuracy, tone, and formatting. I handled multilingual quality checks and RLHF feedback to support LLM training and ensured high output quality following detailed rubrics. • Transcribed and annotated both English and Korean audio data with specialized prosodic and disfluency tags • Rated AI model outputs for factual correctness, tone, formatting, and instruction adherence • Conducted culturally sensitive checks and translation QA for multilingual data • Managed high-volume labeling and evaluation workflows efficiently