Melissa Moncrief

Key Skills

Software

Label Studio

Prodigy

Top Subject Matter

Ecommerce- customer support

Design- digital graphics

Business- marketing

Top Data Types

Image

Video

Text

Top Task Types

Question Answering

Text Generation

Evaluation/Rating

Data Collection

Freelancer Overview

I am a PhD-trained data annotation specialist with over six years of experience supporting machine learning and large language model (LLM) development. My expertise spans high-quality data labeling, annotation guideline design, and rigorous evaluation of AI-generated outputs for reasoning, factual accuracy, and policy compliance. I have annotated and reviewed over 500,000 data points for NLP and supervised learning tasks, maintaining a >98% accuracy rate across multiple large-scale projects. Skilled in Python, SQL, and tools like Label Studio, Prodigy, and CVAT, I excel at producing gold-standard datasets, conducting QA audits, and resolving complex labeling cases. My background also includes designing text classification datasets, leading annotation quality audits, and contributing to peer-reviewed AI research. I am committed to delivering reliable, high-quality training data that drives robust and ethical AI systems.

Labeling Experience

Senior Data Annotation & AI Training Specialist

Label StudioTextEvaluation Rating

As Senior Data Annotation & AI Training Specialist, I led the annotation and review of over 500,000 data points for training and evaluating large language models and supervised ML systems. My tasks included producing gold-standard labels for complex reasoning, mathematics, and factuality tasks, and assessing AI responses against criteria such as correctness, reasoning depth, and policy compliance. I also conducted inter-annotator agreement reviews and managed the timely and accurate resolution of edge-case labeling disputes. • Maintained over 98% annotation accuracy across long-term projects. • Ensured timely delivery within high-volume, remote environments. • Leveraged software such as Label Studio, Prodigy, and CVAT to label and review data. • Oversaw QA audits and contributed to annotation guideline refinement.

As Senior Data Annotation & AI Training Specialist, I led the annotation and review of over 500,000 data points for training and evaluating large language models and supervised ML systems. My tasks included producing gold-standard labels for complex reasoning, mathematics, and factuality tasks, and assessing AI responses against criteria such as correctness, reasoning depth, and policy compliance. I also conducted inter-annotator agreement reviews and managed the timely and accurate resolution of edge-case labeling disputes. • Maintained over 98% annotation accuracy across long-term projects. • Ensured timely delivery within high-volume, remote environments. • Leveraged software such as Label Studio, Prodigy, and CVAT to label and review data. • Oversaw QA audits and contributed to annotation guideline refinement.

2021

AI Research Assistant / Data Scientist

ProdigyTextClassification

As an AI Research Assistant / Data Scientist, I supported machine learning research initiatives by performing data annotation and preprocessing for NLP classification and regression tasks. I systematically built and validated annotated datasets focused on natural language processing. I conducted comprehensive error analysis to uncover and document systematic model weaknesses. • Designed annotation schemas and validation routines. • Worked extensively with labeled text and tabular data. • Utilized tools such as Label Studio and Prodigy to annotate and review data. • Authored technical documentation for annotation processes.

As an AI Research Assistant / Data Scientist, I supported machine learning research initiatives by performing data annotation and preprocessing for NLP classification and regression tasks. I systematically built and validated annotated datasets focused on natural language processing. I conducted comprehensive error analysis to uncover and document systematic model weaknesses. • Designed annotation schemas and validation routines. • Worked extensively with labeled text and tabular data. • Utilized tools such as Label Studio and Prodigy to annotate and review data. • Authored technical documentation for annotation processes.

2017 - 2021

Education

Work History