Ashley Peters

Key Skills

Software

Scale AI

Other

Top Subject Matter

Code Evaluation

LLM Training

Software Engineering

Top Data Types

Text

Audio

Image

Document

Top Task Types

RLHF

Prompt Response Writing SFT

Transcription

Classification

Freelancer Overview

Senior AI Code Evaluator & LLM Trainer. Brings 2+ years of professional experience across legal operations, contract review, compliance, and structured analysis. Core strengths include Scale AI, Handshake AI, and Internal. Education includes Master of Science, Carnegie Mellon University (2024) and Bachelor of Science, University of Tennessee, Knoxville (2022). AI-training focus includes data types such as Computer Code, Programming, and Text and labeling workflows including RLHF, Prompt + Response Writing (SFT), and Transcription.

Labeling Experience

Senior AI Code Evaluator & LLM Trainer

Scale AIRLHF

As a Senior AI Code Evaluator & LLM Trainer at Outlier (Scale AI), I evaluate AI-generated code outputs for quality and alignment. I provide structured RLHF preference feedback to enhance LLM reasoning and code generation, designing adversarial prompts and analyzing model outputs for failure modes. My work directly feeds into fine-tuning datasets for major AI labs, focusing on code correctness, security, and instruction adherence. • Evaluated 250+ code outputs monthly across Python, JavaScript, Java, and C++ • Delivered RLHF feedback and comparative output rankings • Designed red-teaming and edge-case test scenarios • Maintained a 98% quality score as a top 5% evaluator

As a Senior AI Code Evaluator & LLM Trainer at Outlier (Scale AI), I evaluate AI-generated code outputs for quality and alignment. I provide structured RLHF preference feedback to enhance LLM reasoning and code generation, designing adversarial prompts and analyzing model outputs for failure modes. My work directly feeds into fine-tuning datasets for major AI labs, focusing on code correctness, security, and instruction adherence. • Evaluated 250+ code outputs monthly across Python, JavaScript, Java, and C++ • Delivered RLHF feedback and comparative output rankings • Designed red-teaming and edge-case test scenarios • Maintained a 98% quality score as a top 5% evaluator

2024 - Present

Freelance AI Trainer & Data Annotator

TextPrompt Response Writing SFT

As a Freelance AI Trainer & Data Annotator with Handshake AI, Surge AI, and DataAnnotation.tech, I completed diverse NLP and AI data labeling assignments. My tasks included annotation, preference ranking, instruction-following evaluation, and conversational prompt engineering for STEM and software domains. I contributed to multimodal AI pipelines, creating datasets for conversational and content moderation models. • Labeled and reviewed 600+ NLP, ranking, and moderation tasks • Developed instruction-tuning and prompt-response datasets • Worked across text, image, audio, and video annotation pipelines • Achieved consistent 5-star quality and top contributor status

As a Freelance AI Trainer & Data Annotator with Handshake AI, Surge AI, and DataAnnotation.tech, I completed diverse NLP and AI data labeling assignments. My tasks included annotation, preference ranking, instruction-following evaluation, and conversational prompt engineering for STEM and software domains. I contributed to multimodal AI pipelines, creating datasets for conversational and content moderation models. • Labeled and reviewed 600+ NLP, ranking, and moderation tasks • Developed instruction-tuning and prompt-response datasets • Worked across text, image, audio, and video annotation pipelines • Achieved consistent 5-star quality and top contributor status

2023 - 2024

Graduate Research Assistant – AI Systems Lab

TextRLHF

As a Graduate Research Assistant at Carnegie Mellon AI Systems Lab, I evaluated LLM outputs and performed RLHF on code generation models. My thesis involved building evaluation frameworks to assess alignment between LLM outputs and human expert preferences. I contributed to research-centric training datasets and model assessment tasks. • Built RLHF protocols for advanced code generation alignment • Designed evaluation frameworks for LLM output benchmarking • Supported dataset creation and fine-tuning cycles • Presented research findings at academic venues

As a Graduate Research Assistant at Carnegie Mellon AI Systems Lab, I evaluated LLM outputs and performed RLHF on code generation models. My thesis involved building evaluation frameworks to assess alignment between LLM outputs and human expert preferences. I contributed to research-centric training datasets and model assessment tasks. • Built RLHF protocols for advanced code generation alignment • Designed evaluation frameworks for LLM output benchmarking • Supported dataset creation and fine-tuning cycles • Presented research findings at academic venues

2022 - 2024

Image Annotation Contributor – Open Source

OtherImageClassification

I produced and labeled over 150 image annotation datasets for computer vision model training, focusing on 2D and 3D content for AI. My work utilized digital art tools to create synthetic and real image datasets for classification tasks. These contributions supported computer vision benchmarks and model validation. • Generated and annotated 2D/3D images for vision models • Used GIMP, Blender, and industry tools for dataset production • Focused on accurate classification labeling and diversity • Contributed datasets to open-source research projects

I produced and labeled over 150 image annotation datasets for computer vision model training, focusing on 2D and 3D content for AI. My work utilized digital art tools to create synthetic and real image datasets for classification tasks. These contributions supported computer vision benchmarks and model validation. • Generated and annotated 2D/3D images for vision models • Used GIMP, Blender, and industry tools for dataset production • Focused on accurate classification labeling and diversity • Contributed datasets to open-source research projects

2023 - 2023

Audio Dataset Contributor – Open Source

OtherAudioTranscription

I produced over 200 audio transcription and sound classification samples for speech AI model training. I recorded, transcribed, and labeled audio data using industry-standard tools for open-source datasets. My contributions enhanced the quality and diversity of speech AI training sets in collaborative projects. • Created diverse speech and sound sample recordings • Performed manual transcription and labeling • Applied detailed classification for audio features • Supported open-source model training efforts

I produced over 200 audio transcription and sound classification samples for speech AI model training. I recorded, transcribed, and labeled audio data using industry-standard tools for open-source datasets. My contributions enhanced the quality and diversity of speech AI training sets in collaborative projects. • Created diverse speech and sound sample recordings • Performed manual transcription and labeling • Applied detailed classification for audio features • Supported open-source model training efforts

2023 - 2023

Show more

Education

Work History