For employers

Hire this AI Trainer

Sign in or create an account to invite AI Trainers to your job.

LoginCreate account
Invite to Job
S

Shruti Sharma

AI Driven Data Analyst (LLM Evaluation and RLHF Data Labeling)

India flagU.P., India
IntermediateOther

Key Skills

Software

Other

Top Subject Matter

Pharmaceutical sales data and AI model evaluation
Transactional risk and AI output evaluation
Large Language Models

Top Data Types

TextText
ImageImage
DocumentDocument

Top Task Types

RLHFRLHF
Text GenerationText Generation
Data CollectionData Collection

Freelancer Overview

AI Driven Data Analyst (LLM Evaluation and RLHF Data Labeling). Brings 3+ years of professional experience across complex professional workflows, research, and quality-focused execution. Core strengths include Internal, Proprietary Tooling, and Other. Education includes Doctor of Philosophy, N/A (2025) and Master of Science, Syracuse University (2018). AI-training focus includes data types such as Text and labeling workflows including RLHF, Evaluation, and Rating.

Intermediate

Labeling Experience

AI Driven Data Analyst (LLM Evaluation and RLHF Data Labeling)

TextRLHF
As an AI Driven Data Analyst at Nirmal Sales, contributed structured RLHF feedback for large language models by ranking, correcting, and documenting AI-generated outputs. Evaluated discrepancies between AI and human-labeled decisions to identify systemic errors and ambiguity. Maintained annotation records, ensured data quality, and followed detailed labeling guidelines for improved LLM evaluation workflows. • Provided feedback to improve evaluation metrics and human-in-the-loop validation. • Tracked validation outcomes and maintained high-quality datasets using Excel and spreadsheets. • Used Python and SQL for analysis of labeling outcomes and error identification. • Ensured compliance with process guidelines and supported AI content safety annotation.

As an AI Driven Data Analyst at Nirmal Sales, contributed structured RLHF feedback for large language models by ranking, correcting, and documenting AI-generated outputs. Evaluated discrepancies between AI and human-labeled decisions to identify systemic errors and ambiguity. Maintained annotation records, ensured data quality, and followed detailed labeling guidelines for improved LLM evaluation workflows. • Provided feedback to improve evaluation metrics and human-in-the-loop validation. • Tracked validation outcomes and maintained high-quality datasets using Excel and spreadsheets. • Used Python and SQL for analysis of labeling outcomes and error identification. • Ensured compliance with process guidelines and supported AI content safety annotation.

2024 - 2025

AI Driven Data Analyst (AI Output Annotation and Evaluation)

Text
As an AI Driven Data Analyst at SasInfo, was responsible for evaluating AI-assisted outputs and transactional data for safety, relevance, and policy adherence. Created structured annotations, justifications, and comparative assessments aligning with annotation and grading workflows. Provided ongoing feedback to refine AI model outputs and support iterative improvement. • Conducted large-scale case annotation, documenting edge cases and inconsistencies. • Applied structured, SME-style grading and annotation principles to content evaluation. • Used SQL and Python for risk assessment and fraud detection in labeled datasets. • Maintained accuracy and consistency across high-volume AI evaluation tasks.

As an AI Driven Data Analyst at SasInfo, was responsible for evaluating AI-assisted outputs and transactional data for safety, relevance, and policy adherence. Created structured annotations, justifications, and comparative assessments aligning with annotation and grading workflows. Provided ongoing feedback to refine AI model outputs and support iterative improvement. • Conducted large-scale case annotation, documenting edge cases and inconsistencies. • Applied structured, SME-style grading and annotation principles to content evaluation. • Used SQL and Python for risk assessment and fraud detection in labeled datasets. • Maintained accuracy and consistency across high-volume AI evaluation tasks.

2023 - 2024

Researcher (Predictive Model Annotation and Data Labeling)

OtherTextData Collection
Engineered predictive analysis pipelines incorporating labeling, feature engineering, and annotation to structured real estate datasets for model development. Annotated and evaluated AI outputs, categorized errors, and generated RLHF-aligned evaluation signals to improve data quality and model interpretability. Delivered human-readable justifications and clear documentation for each annotated case supporting the modeling process. • Handled structured datasets using Python, Excel, and SQL. • Applied segmentation, labeling, and annotation for urban housing price prediction models. • Documented edge cases, anomalies, and rationales for evaluation tasks. • Supported model validation through annotated data and performance analysis.

Engineered predictive analysis pipelines incorporating labeling, feature engineering, and annotation to structured real estate datasets for model development. Annotated and evaluated AI outputs, categorized errors, and generated RLHF-aligned evaluation signals to improve data quality and model interpretability. Delivered human-readable justifications and clear documentation for each annotated case supporting the modeling process. • Handled structured datasets using Python, Excel, and SQL. • Applied segmentation, labeling, and annotation for urban housing price prediction models. • Documented edge cases, anomalies, and rationales for evaluation tasks. • Supported model validation through annotated data and performance analysis.

Not specified

Generative AI & LLM Evaluation Specialist (Prompt and Output Annotation)

OtherTextText Generation
Contributed to multiple Generative AI & LLM Evaluation projects by researching, designing, and evaluating prompts and outputs for GPT-powered tools. Performed structured prompt testing, ranking, rewriting, and analysis of AI-generated responses for quality, safety, and relevance. Documented findings in standardized formats following RLHF and annotation guidelines. • Generated and evaluated prompts for Copy.ai, Algolia, Debuild, Viable, and Epsilon Code. • Assessed response quality, tone, hallucination risk, and guideline adherence in various AI solutions. • Compared model behaviors, identified edge cases, and contributed to guideline refinement. • Used structured rationales for feedback supporting iterative model training.

Contributed to multiple Generative AI & LLM Evaluation projects by researching, designing, and evaluating prompts and outputs for GPT-powered tools. Performed structured prompt testing, ranking, rewriting, and analysis of AI-generated responses for quality, safety, and relevance. Documented findings in standardized formats following RLHF and annotation guidelines. • Generated and evaluated prompts for Copy.ai, Algolia, Debuild, Viable, and Epsilon Code. • Assessed response quality, tone, hallucination risk, and guideline adherence in various AI solutions. • Compared model behaviors, identified edge cases, and contributed to guideline refinement. • Used structured rationales for feedback supporting iterative model training.

Not specified

Education

S

Syracuse University

Master of Science, Computer Engineering

Master of Science
2017 - 2018
U

U.P.T.U.

Bachelor of Technology, Electronics and Communication Engineering

Bachelor of Technology
2007 - 2011

Work History

N

Nirmal Sales

Data Analyst

U.P.
2024 - 2025
S

SasInfo

Data Analyst

Charlotte
2023 - 2024