Aayush Kumar

Key Skills

Software

Google Cloud Vertex AI

Mercor

Other

Internal/Proprietary Tooling

Top Subject Matter

No subject matter listed

Top Data Types

Computer Code Programming

Document

Image

Text

Video

Top Label Types

Evaluation Rating

Fine Tuning

Prompt Response Writing SFT

RLHF

Freelancer Overview

I have hands-on experience in AI data operations, data annotation, and managing large-scale AI training data projects. My background includes leading teams in the creation and quality assurance of RLHF, CUA, and SFT datasets for large language models, ensuring high annotation standards and strict guideline compliance. I have worked directly on structured data labeling, technical prompt annotation, and refining annotation guidelines to improve accuracy and consistency for LLM training and benchmarking. My technical skills span Python, shell scripting, AWS, and cloud-based data workflows, and I have contributed to computer vision projects such as vehicle detection using CNNs. With experience coordinating teams of over 250 contributors and optimizing data pipelines, I am adept at delivering high-quality training data for enterprise AI applications.

Labeling Experience

Delivery Manager

Internal Proprietary ToolingTextRLHFFine Tuning

– Started as a trainer for Codeforces-style competitive programming tasks, evaluating contributor solutions and guiding improvements in algorithmic reasoning and code quality. – Worked on the OSWorld RLHF dataset and later managed RL data operations for a team of 20 contributors focused on reinforcement learning training data. – Led development of CUA and SFT datasets used for training large language models while ensuring annotation quality and guideline compliance. – Promoted to Delivery Manager managing 250+ contributors and 10 POD Leads while coordinating dataset production for enterprise clients including Alibaba. – Designed task distribution pipelines, review workflows, and quality control systems to maintain high quality LLM training data.

– Started as a trainer for Codeforces-style competitive programming tasks, evaluating contributor solutions and guiding improvements in algorithmic reasoning and code quality. – Worked on the OSWorld RLHF dataset and later managed RL data operations for a team of 20 contributors focused on reinforcement learning training data. – Led development of CUA and SFT datasets used for training large language models while ensuring annotation quality and guideline compliance. – Promoted to Delivery Manager managing 250+ contributors and 10 POD Leads while coordinating dataset production for enterprise clients including Alibaba. – Designed task distribution pipelines, review workflows, and quality control systems to maintain high quality LLM training data.

2025 - 2025

LLM Researcher

MercorDocumentEvaluation Rating

– Performed structured data annotation & evaluation for datasets used in training & benchmarking large language models. – Annotated technical prompts, responses, and reasoning chains to improve dataset accuracy and model alignment. – Worked with reviewers to refine annotation guidelines & maintain consistency across large scale labeling tasks.

– Performed structured data annotation & evaluation for datasets used in training & benchmarking large language models. – Annotated technical prompts, responses, and reasoning chains to improve dataset accuracy and model alignment. – Worked with reviewers to refine annotation guidelines & maintain consistency across large scale labeling tasks.

2025 - 2025

Education

Work History