Anthony Pastrana

Key Skills

Software

CrowdSource

Mercor

OneForma

Scale AI

Top Subject Matter

No subject matter listed

Top Data Types

Audio

Computer Code Programming

Document

Geospatial Tiled Imagery

Image

Text

Video

Top Label Types

Action Recognition

Audio Recording

Computer Programming Coding

Data Collection

Emotion Recognition

Evaluation Rating

Fine Tuning

Prompt Response Writing SFT

RLHF

Segmentation

Text Summarization

Transcription

Freelancer Overview

I am an experienced AI operations specialist with a strong background in data labeling, annotation, and model training for large language models and AI systems. My expertise includes managing remote teams of annotators and auditors, developing detailed rubrics and instructional materials, and optimizing workflows to ensure high-quality training data. I have led cross-functional projects at Scale AI and Mercor, where I created comprehensive dashboards, designed bot agents for annotator support, and analyzed thousands of task responses to refine benchmarks and training approaches. My technical skills span machine learning, performance management, and systems administration, and I am adept at collaborating with stakeholders to deliver accurate, scalable AI data solutions. I am passionate about leveraging technology, including AI chatbots and automation tools, to streamline operations and drive continuous improvement in data quality and project delivery.

Labeling Experience

AI Auditor Training & Data Quality Benchmarking

Scale AITextData Collection

Led and supported AI data quality initiatives by training and auditing remote annotation teams. Developed detailed rubrics and grading instructions for multiple writing and editing task types. Analyzed thousands of annotated responses and auditor scores to establish benchmarks, identify quality gaps, and improve consistency across AI training datasets.

Led and supported AI data quality initiatives by training and auditing remote annotation teams. Developed detailed rubrics and grading instructions for multiple writing and editing task types. Analyzed thousands of annotated responses and auditor scores to establish benchmarks, identify quality gaps, and improve consistency across AI training datasets.

2024 - 2025

LLM Data Annotation & Quality Evaluation Project

Scale AITextRLHF

Worked on large-scale text annotation and evaluation projects supporting the training of large language models. Reviewed, labeled, and evaluated model-generated responses for quality, relevance, and instruction adherence. Collaborated with cross-functional teams to refine task guidelines, improve prompt clarity, and reduce annotation inconsistencies. Contributed feedback used to improve data quality and downstream model performance.

Worked on large-scale text annotation and evaluation projects supporting the training of large language models. Reviewed, labeled, and evaluated model-generated responses for quality, relevance, and instruction adherence. Collaborated with cross-functional teams to refine task guidelines, improve prompt clarity, and reduce annotation inconsistencies. Contributed feedback used to improve data quality and downstream model performance.

2023 - 2025

Education

Work History