For employers

Hire this AI Trainer

Sign in or create an account to invite AI Trainers to your job.

LoginCreate account
Invite to Job
Yash Sonawane

Yash Sonawane

LLM Output Evaluator & Prompt Engineer – PostmortemAI

INDIA flag
Remote, India
$8.00/hrEntry LevelOtherScale AILabelbox

Key Skills

Software

Other
Scale AIScale AI
LabelboxLabelbox

Top Subject Matter

Incident postmortem reporting
LLM behavior evaluation
DevOps incidents

Top Data Types

TextText
VideoVideo
DocumentDocument

Top Task Types

RLHF
Prompt Response Writing SFT

Freelancer Overview

LLM Output Evaluator & Prompt Engineer – PostmortemAI. Brings 1+ years of professional experience across complex professional workflows, research, and quality-focused execution. Core strengths include Internal, Proprietary Tooling, and Other. Education includes Bachelor of Technology, N/A (2027). AI-training focus includes data types such as Text and labeling workflows including RLHF, Prompt + Response Writing (SFT), and Evaluation.

Entry LevelHindiEnglish

Labeling Experience

Freelance AI Workflow Specialist – Independent Clients (Upwork)

OtherTextPrompt Response Writing SFT
As a Freelance AI Workflow Specialist for independent clients, I delivered labeled datasets for LLM fine-tuning across technical domains. I performed safety checks, quality reviews, and created annotation guidelines for distributed teams. I maintained high annotation speed and quality consistency, ensuring minimal revisions. • Produced prompt-response training data under detailed instruction rubrics. • Reviewed and flagged bias, accuracy, and tone issues in AI-generated texts. • Authored annotation instructions and labeling schemas for client teams. • Optimized batch labeling throughput with strong accuracy metrics.

As a Freelance AI Workflow Specialist for independent clients, I delivered labeled datasets for LLM fine-tuning across technical domains. I performed safety checks, quality reviews, and created annotation guidelines for distributed teams. I maintained high annotation speed and quality consistency, ensuring minimal revisions. • Produced prompt-response training data under detailed instruction rubrics. • Reviewed and flagged bias, accuracy, and tone issues in AI-generated texts. • Authored annotation instructions and labeling schemas for client teams. • Optimized batch labeling throughput with strong accuracy metrics.

2024 - Present

LLM Output Evaluator & Prompt Engineer – PostmortemAI

TextRLHF
As an LLM Output Evaluator & Prompt Engineer for PostmortemAI, I iteratively designed and rated model outputs to enhance response quality. I manually reviewed and scored LLM-generated text for correctness, tone, and adherence to instructions. I developed labeling schemas and structured output examples to guide model improvements. • Designed and evaluated hundreds of prompts and few-shot examples for model training. • Rated and annotated outputs for quality, completeness, and safety. • Established structured evaluation protocols and feedback cycles. • Identified critical failure modes such as hallucination and instruction drift.

As an LLM Output Evaluator & Prompt Engineer for PostmortemAI, I iteratively designed and rated model outputs to enhance response quality. I manually reviewed and scored LLM-generated text for correctness, tone, and adherence to instructions. I developed labeling schemas and structured output examples to guide model improvements. • Designed and evaluated hundreds of prompts and few-shot examples for model training. • Rated and annotated outputs for quality, completeness, and safety. • Established structured evaluation protocols and feedback cycles. • Identified critical failure modes such as hallucination and instruction drift.

2024 - Present

Technical Content Annotator & Writer – DEV.to / CodeTreasureHub

OtherTextPrompt Response Writing SFT
As a Technical Content Annotator & Writer at DEV.to / CodeTreasureHub, I produced structured prompt-response pairs and technical content for instruction fine-tuning datasets. I reviewed and edited AI-generated drafts for technical accuracy, hallucinations, and compliance. I consistently applied style and formatting guidelines similar to those used in annotation workflows. • Authored and annotated 60+ technical articles on DevOps and AI tools. • Edited and reviewed LLM-generated outputs for quality and factual correctness. • Created and followed detailed formatting standards for annotation discipline. • Produced datasets and tutorials mimicking instruction-tuning data formats.

As a Technical Content Annotator & Writer at DEV.to / CodeTreasureHub, I produced structured prompt-response pairs and technical content for instruction fine-tuning datasets. I reviewed and edited AI-generated drafts for technical accuracy, hallucinations, and compliance. I consistently applied style and formatting guidelines similar to those used in annotation workflows. • Authored and annotated 60+ technical articles on DevOps and AI tools. • Edited and reviewed LLM-generated outputs for quality and factual correctness. • Created and followed detailed formatting standards for annotation discipline. • Produced datasets and tutorials mimicking instruction-tuning data formats.

2023 - Present

AI Automation & Workflow Tester – NotionOps AI (Hackathon)

Text
As an AI Automation & Workflow Tester for NotionOps AI during a hackathon, I evaluated conversational flows and annotated webhook payloads to improve AI summarization. I stress-tested prompts and classified diverse inputs to surface response failures. I documented findings in structured evaluation logs to guide future routing logic enhancements. • Designed and rated Claude API responses for correctness and consistency. • Annotated GitHub webhook payloads for model training purposes. • Tested edge cases and adversarial inputs to identify model weaknesses. • Recorded annotation results for iterative workflow improvements.

As an AI Automation & Workflow Tester for NotionOps AI during a hackathon, I evaluated conversational flows and annotated webhook payloads to improve AI summarization. I stress-tested prompts and classified diverse inputs to surface response failures. I documented findings in structured evaluation logs to guide future routing logic enhancements. • Designed and rated Claude API responses for correctness and consistency. • Annotated GitHub webhook payloads for model training purposes. • Tested edge cases and adversarial inputs to identify model weaknesses. • Recorded annotation results for iterative workflow improvements.

2024 - 2024

Education

N

N/A

Bachelor of Technology, Electrical Engineering

Bachelor of Technology
2023 - 2027

Work History

U

Upwork

DevOps and LLM Engineer Freelancer

Remote
2024 - Present
I

Independent

Open Source Contributor and Hackathon Builder

Pune
2023 - Present