Medical AI Evaluator and Annotation Specialist (Outlier, LLMs)
OtherText
Reviewed outputs from large language models for clinical reasoning, factuality, and safety within health content. Used established rubrics and guidelines to score and annotate AI-generated medical and public health responses. Identified errors, unsafe recommendations, and ambiguous outputs, providing structured feedback for model improvement.
• Consistently assessed AI outputs on instruction-following, accuracy, and risk.
• Applied clinical expertise to scenario evaluation and annotation tasks.
• Ensured adherence to medical guidelines during AI system evaluation.
• Provided written justifications supporting scoring rationale.
Reviewed outputs from large language models for clinical reasoning, factuality, and safety within health content. Used established rubrics and guidelines to score and annotate AI-generated medical and public health responses. Identified errors, unsafe recommendations, and ambiguous outputs, providing structured feedback for model improvement. • Consistently assessed AI outputs on instruction-following, accuracy, and risk. • Applied clinical expertise to scenario evaluation and annotation tasks. • Ensured adherence to medical guidelines during AI system evaluation. • Provided written justifications supporting scoring rationale.
2022 - Present