Shivam Gupta

Key Skills

Software

Mercor

OneForma

Appen

Top Subject Matter

LLM Evaluation

RLHF Domain Expertise

IT Domain

Top Data Types

Text

Audio

Document

Top Task Types

Transcription

RLHF

Segmentation

Text Generation

Text Summarization

Evaluation/Rating

Fine-tuning

Red Teaming

Freelancer Overview

AI Content Specialist / RLHF Generalist. Brings 4+ years of professional experience across complex professional workflows, research, and quality-focused execution. Core strengths include Mercor, OneForma, and Appen. Education includes Bachelor of Science, University of Delhi (2023). AI-training focus includes data types such as Text and Audio and labeling workflows including Evaluation, Rating, and Transcription.

Labeling Experience

AI Content Specialist / RLHF Generalist

MercorText

As an AI Content Specialist and RLHF Generalist at Mercor, I performed English-language LLM evaluation using side-by-side comparison, response ranking, and prompt-based scoring. I participated in high-volume evaluation sprints and a high-priority IT-domain sprint focused on accuracy, safety, and alignment. I evaluated outputs for truthfulness, helpfulness, and safety, and contributed to AI alignment and response quality improvement. • Conducted trial AI red teaming tasks for edge-case prompts. • Assessed technical and general LLM responses for relevance and correctness. • Delivered consistent annotations under strict guidelines. • Supported model improvement by ranking and scoring outputs.

As an AI Content Specialist and RLHF Generalist at Mercor, I performed English-language LLM evaluation using side-by-side comparison, response ranking, and prompt-based scoring. I participated in high-volume evaluation sprints and a high-priority IT-domain sprint focused on accuracy, safety, and alignment. I evaluated outputs for truthfulness, helpfulness, and safety, and contributed to AI alignment and response quality improvement. • Conducted trial AI red teaming tasks for edge-case prompts. • Assessed technical and general LLM responses for relevance and correctness. • Delivered consistent annotations under strict guidelines. • Supported model improvement by ranking and scoring outputs.

2025 - Present

Voice AI Data Annotation & Review

OneformaAudioTranscription

For voice AI data annotation and review projects, I transcribed and annotated over 800 audio samples (around 80 hours) for voice assistant systems. I was promoted to Reviewer, ensuring high-quality dataset consistency within multilingual and context-sensitive settings. I improved voice assistant model training by maintaining rigorous validation and guideline enforcement. • Conducted multilingual data annotation (Hindi–English). • Maintained high annotation reliability for voice AI. • Supported training data creation for speech systems. • Upheld rigorous quality procedures for annotation and review.

For voice AI data annotation and review projects, I transcribed and annotated over 800 audio samples (around 80 hours) for voice assistant systems. I was promoted to Reviewer, ensuring high-quality dataset consistency within multilingual and context-sensitive settings. I improved voice assistant model training by maintaining rigorous validation and guideline enforcement. • Conducted multilingual data annotation (Hindi–English). • Maintained high annotation reliability for voice AI. • Supported training data creation for speech systems. • Upheld rigorous quality procedures for annotation and review.

2025 - 2025

Biology-Based Prompt Evaluator

MercorText

In Biology-based prompt evaluation projects, I assessed AI responses for STEM concepts, factual accuracy, and scientific reasoning. I verified outputs for logical consistency and domain correctness, supporting model improvement in Biology and STEM Q&A. I contributed to higher model performance on scientific evaluations. • Evaluated complex Biology concepts in LLM outputs. • Ensured scientific rigor in prompt-based evaluation. • Improved STEM-specific model response accuracy. • Enhanced domain-aligned reasoning in AI.

In Biology-based prompt evaluation projects, I assessed AI responses for STEM concepts, factual accuracy, and scientific reasoning. I verified outputs for logical consistency and domain correctness, supporting model improvement in Biology and STEM Q&A. I contributed to higher model performance on scientific evaluations. • Evaluated complex Biology concepts in LLM outputs. • Ensured scientific rigor in prompt-based evaluation. • Improved STEM-specific model response accuracy. • Enhanced domain-aligned reasoning in AI.

2025 - 2025

Technical LLM Evaluator (IT Domain Sprint)

MercorText

In a high-priority technical LLM evaluation sprint with Mercor, I evaluated AI outputs for technical and IT-related prompts in a demanding 24-hour timeframe. I ensured high-accuracy validation and rapid response ranking under strict deadline pressure. I assessed outputs for technical correctness and relevance. • Supported IT-domain LLM improvement. • Delivered fast, high-quality evaluation in time-sensitive settings. • Verified technical responses for domain expertise. • Maintained accuracy and guideline adherence throughout sprint.

In a high-priority technical LLM evaluation sprint with Mercor, I evaluated AI outputs for technical and IT-related prompts in a demanding 24-hour timeframe. I ensured high-accuracy validation and rapid response ranking under strict deadline pressure. I assessed outputs for technical correctness and relevance. • Supported IT-domain LLM improvement. • Delivered fast, high-quality evaluation in time-sensitive settings. • Verified technical responses for domain expertise. • Maintained accuracy and guideline adherence throughout sprint.

2025 - 2025

AI Data & Localization Specialist

OneformaAudioTranscription

As an AI Data & Localization Specialist at OneForma, I transcribed and annotated approximately 80 hours of audio data for voice assistant and speech recognition training. I was promoted to Reviewer, where I validated contributor outputs to ensure dataset quality and annotation consistency. I worked with Hindi–English datasets, maintaining high-quality training data through guideline adherence. • Reviewed and evaluated multilingual datasets for accuracy. • Ensured annotation compliance and contextual consistency. • Upheld dataset quality through rigorous quality checks. • Improved voice AI systems via reliable data annotation.

As an AI Data & Localization Specialist at OneForma, I transcribed and annotated approximately 80 hours of audio data for voice assistant and speech recognition training. I was promoted to Reviewer, where I validated contributor outputs to ensure dataset quality and annotation consistency. I worked with Hindi–English datasets, maintaining high-quality training data through guideline adherence. • Reviewed and evaluated multilingual datasets for accuracy. • Ensured annotation compliance and contextual consistency. • Upheld dataset quality through rigorous quality checks. • Improved voice AI systems via reliable data annotation.

2025 - 2025

Show more

Education

Work History