Emily Peters

Key Skills

Software

Labelbox

Scale AI

Prodigy

Top Subject Matter

Healthcare/Clinical AI

Llms Domain Expertise

Healthcare/Medical AI

Top Data Types

Image

Video

Text

Audio

Document

Top Label Types

Bounding Box

Object Detection

Tracking

Entity Ner Classification

Classification

Question Answering

Text Generation

RLHF

Emotion Recognition

Evaluation Rating

Transcription

Freelancer Overview

Senior AI Trainer & AI Quality Lead. Brings 4+ years of professional experience across complex professional workflows, research, and quality-focused execution. Core strengths include Internal, Proprietary Tooling, and Labelbox. Education includes Bachelor of Science, Oregon State University (2017) and Diploma, Oregon Health & Science College (2019). AI-training focus includes data types such as Text and labeling workflows including Evaluation, Rating, and Classification.

Labeling Experience

Senior AI Trainer & AI Quality Lead

Text

Led structured large language model (LLM) evaluation initiatives, focusing on increasing response accuracy and refining reward modeling. Designed and implemented model performance tracking for annotation consistency and medical safety alignment. Collaborated with ML engineers to improve RLHF scoring criteria and evaluation rubrics. • Led adversarial testing and safety audits to reduce high-risk outputs • Built KPI dashboards for annotation throughput and edge-case failures • Improved structured feedback loops for model optimization • Increased model response accuracy by 22%

Led structured large language model (LLM) evaluation initiatives, focusing on increasing response accuracy and refining reward modeling. Designed and implemented model performance tracking for annotation consistency and medical safety alignment. Collaborated with ML engineers to improve RLHF scoring criteria and evaluation rubrics. • Led adversarial testing and safety audits to reduce high-risk outputs • Built KPI dashboards for annotation throughput and edge-case failures • Improved structured feedback loops for model optimization • Increased model response accuracy by 22%

2023 - Present

Healthcare Video Annotation & AI Training Specialist

LabelboxVideoBounding BoxObject Detection

Led video annotation projects for healthcare-focused AI systems, labeling clinical procedures, patient interactions, and medical device usage for training computer vision models. Applied bounding boxes and object tracking to identify key anatomical regions, instruments, and actions within video frames. Annotated and tracked multi-step clinical workflows to support AI action recognition and reinforcement learning pipelines. Maintained high-quality annotation standards through inter-annotator calibration, achieving 98%+ accuracy across datasets. Collaborated with ML engineers to integrate annotated videos into YOLO-based and other object detection pipelines, monitoring model performance and refining annotations based on model feedback. Developed KPI dashboards to track annotation throughput, consistency, and error rates, ensuring scalable and reproducible labeling workflows.

Led video annotation projects for healthcare-focused AI systems, labeling clinical procedures, patient interactions, and medical device usage for training computer vision models. Applied bounding boxes and object tracking to identify key anatomical regions, instruments, and actions within video frames. Annotated and tracked multi-step clinical workflows to support AI action recognition and reinforcement learning pipelines. Maintained high-quality annotation standards through inter-annotator calibration, achieving 98%+ accuracy across datasets. Collaborated with ML engineers to integrate annotated videos into YOLO-based and other object detection pipelines, monitoring model performance and refining annotations based on model feedback. Developed KPI dashboards to track annotation throughput, consistency, and error rates, ensuring scalable and reproducible labeling workflows.

2022

Healthcare & Object Detection Data Annotation Specialist

LabelboxImageBounding BoxObject Detection

Led a healthcare-focused computer vision annotation project, labeling medical imaging datasets for training object detection models (YOLO). Applied bounding boxes, segmentation masks, and tracking annotations to highlight anatomical regions, medical devices, and clinical findings in imaging data. Ensured annotation consistency and quality by implementing inter-annotator calibration and validation workflows, achieving 98%+ accuracy across datasets. Collaborated closely with ML engineers to prepare annotated datasets for YOLO model training, evaluate model outputs, and refine labeling standards based on model feedback. Monitored annotation throughput, tracked metrics, and maintained detailed documentation of project KPIs to optimize efficiency and maintain high-quality standards.

Led a healthcare-focused computer vision annotation project, labeling medical imaging datasets for training object detection models (YOLO). Applied bounding boxes, segmentation masks, and tracking annotations to highlight anatomical regions, medical devices, and clinical findings in imaging data. Ensured annotation consistency and quality by implementing inter-annotator calibration and validation workflows, achieving 98%+ accuracy across datasets. Collaborated closely with ML engineers to prepare annotated datasets for YOLO model training, evaluate model outputs, and refine labeling standards based on model feedback. Monitored annotation throughput, tracked metrics, and maintained detailed documentation of project KPIs to optimize efficiency and maintain high-quality standards.

2022

Healthcare & Clinical Audio Annotation Specialist

Scale AIAudioQuestion AnsweringText Generation

Led annotation of healthcare-focused audio datasets for training and fine-tuning LLMs and conversational AI models. Transcribed clinical interviews, patient consultations, and treatment discussions, ensuring high fidelity and medical accuracy. Performed emotion recognition and context classification to support AI understanding of tone, intent, and patient sentiment. Curated question-answer pairs and text prompts from audio data to improve model comprehension and response generation. Monitored annotation quality using inter-annotator agreement metrics and implemented calibration workflows to maintain 98%+ accuracy. Collaborated with ML engineers to integrate labeled audio data into RLHF pipelines and model evaluation workflows.

Led annotation of healthcare-focused audio datasets for training and fine-tuning LLMs and conversational AI models. Transcribed clinical interviews, patient consultations, and treatment discussions, ensuring high fidelity and medical accuracy. Performed emotion recognition and context classification to support AI understanding of tone, intent, and patient sentiment. Curated question-answer pairs and text prompts from audio data to improve model comprehension and response generation. Monitored annotation quality using inter-annotator agreement metrics and implemented calibration workflows to maintain 98%+ accuracy. Collaborated with ML engineers to integrate labeled audio data into RLHF pipelines and model evaluation workflows.

2021

Healthcare NLP & LLM Text Annotation Specialist

LabelboxTextEntity Ner ClassificationClassification

Led large-scale text annotation projects for healthcare-focused LLMs, providing high-quality labeled datasets for model training, evaluation, and RLHF pipelines. Performed Named Entity Recognition (NER) on medical entities, classified clinical text, generated question-answer pairs, and curated text for prompt-response training. Applied clinical knowledge to validate terminology, patient guidance, and treatment references, ensuring model outputs were accurate and safe for healthcare applications. Monitored annotation quality with inter-rater reliability metrics, implemented calibration sessions, and tracked key performance indicators (KPIs) for consistency and throughput. Collaborated with ML engineers to refine reward models, optimize prompts, and evaluate model outputs against clinical accuracy and safety standards.

Led large-scale text annotation projects for healthcare-focused LLMs, providing high-quality labeled datasets for model training, evaluation, and RLHF pipelines. Performed Named Entity Recognition (NER) on medical entities, classified clinical text, generated question-answer pairs, and curated text for prompt-response training. Applied clinical knowledge to validate terminology, patient guidance, and treatment references, ensuring model outputs were accurate and safe for healthcare applications. Monitored annotation quality with inter-rater reliability metrics, implemented calibration sessions, and tracked key performance indicators (KPIs) for consistency and throughput. Collaborated with ML engineers to refine reward models, optimize prompts, and evaluate model outputs against clinical accuracy and safety standards.

2021

Show more

Education

Work History