Rory Whitney

Key Skills

Software

Other

Surge AI

Top Subject Matter

AI Training/Data Annotation

NLP and LLM Data Annotation/Evaluation

Legal Services & Contract Review

Top Data Types

Text

Document

Top Task Types

Classification

Freelancer Overview

AI Trainer. Brings 9+ years of professional experience across complex professional workflows, research, and quality-focused execution. Core strengths include Other and Surge AI. Education includes Bachelor of Science, Valdosta State University (2016). AI-training focus includes data types such as Text and labeling workflows including Classification, Evaluation, and Rating.

Labeling Experience

AI Engineer

Surge AIText

As an AI Engineer, I designed NLP data pipelines for annotation systems, processing over 4 million records monthly and standardizing labels for downstream LLM training. I created evaluation harnesses for prompt quality and model ranking, using metrics like precision, recall, and task success rate. I validated structured outputs for entity extraction and summarization tasks to reduce non-deterministic failures. • Worked with internal and client-facing workflows for 15+ projects. • Tuned semantic search and embeddings for knowledge-heavy queries. • Built APIs and tooling for large-scale annotation and review workflows. • Maintained version control for reproducible models and datasets.

As an AI Engineer, I designed NLP data pipelines for annotation systems, processing over 4 million records monthly and standardizing labels for downstream LLM training. I created evaluation harnesses for prompt quality and model ranking, using metrics like precision, recall, and task success rate. I validated structured outputs for entity extraction and summarization tasks to reduce non-deterministic failures. • Worked with internal and client-facing workflows for 15+ projects. • Tuned semantic search and embeddings for knowledge-heavy queries. • Built APIs and tooling for large-scale annotation and review workflows. • Maintained version control for reproducible models and datasets.

2018 - 2022

AI Trainer

OtherTextClassification

As an AI Trainer, I annotated and reviewed over 100,000 examples for text classification, summarization, and intent recognition. I evaluated model outputs against rubric-based benchmarks to improve dataset consistency and flagged hallucinations, ambiguity, and unsafe completions for prompt refinements. I performed manual QA on labeling workflows and documented annotation guidelines for repeatable scoring. • Focused on improving dataset quality for downstream model training. • Used Python and SQL-driven annotation systems for consistent labeling. • Identified and resolved schema issues to reduce correction cycles by 18%. • Authored and maintained annotation guidelines and edge case documentation.

As an AI Trainer, I annotated and reviewed over 100,000 examples for text classification, summarization, and intent recognition. I evaluated model outputs against rubric-based benchmarks to improve dataset consistency and flagged hallucinations, ambiguity, and unsafe completions for prompt refinements. I performed manual QA on labeling workflows and documented annotation guidelines for repeatable scoring. • Focused on improving dataset quality for downstream model training. • Used Python and SQL-driven annotation systems for consistent labeling. • Identified and resolved schema issues to reduce correction cycles by 18%. • Authored and maintained annotation guidelines and edge case documentation.

2016 - 2018

Education

Work History