Barshi Lodhi

Key Skills

Software

Data Annotation Tech

Google Cloud Vertex AI

Redbrick AI

Top Subject Matter

No subject matter listed

Top Data Types

Document

Image

Text

Video

Top Label Types

Classification

Computer Programming Coding

Data Collection

Evaluation Rating

Point Key Point

Prompt Response Writing SFT

Question Answering

Red Teaming

Relationship

RLHF

Segmentation

Text Generation

Text Summarization

Tracking

Freelancer Overview

I am an analytical and detail-oriented professional with hands-on experience in AI training data evaluation, data labeling, and structured data annotation. My work as a Business Analyst and AI Safety Policy Analyst involved performing side-by-side (SxS) output evaluations, fact-checking, and quality assurance for large language model (LLM) outputs, ensuring accuracy, relevance, and adherence to safety standards. I am proficient in using tools like SQL, Power BI, Python, and Excel to automate reporting and visualize data insights, and I have contributed to projects involving financial, healthcare, and customer segmentation data. My strong foundation in data validation, adversarial prompt design (red teaming), and objective rubric development enables me to support high-quality AI workflows and deliver reliable, actionable insights for model improvement.

Labeling Experience

Business Analyst – LLM Output Evaluation

Redbrick AITextEvaluation Rating

As a Business Analyst at Turing, I performed SxS output evaluations to assess accuracy, relevance, and quality of AI system reasoning. My responsibilities included structured fact-checking and data validation to ensure the reliability of LLM outputs. I provided evidence-based analytical justifications and supported quality assurance in AI training workflows. • Conducted side-by-side (SxS) comparisons of LLM outputs for accuracy, safety, and logic. • Evaluated AI-generated content using objective rubrics and standards. • Provided structured feedback for continuous model improvement processes. • Utilized various digital platforms and browser tools for output documentation.

As a Business Analyst at Turing, I performed SxS output evaluations to assess accuracy, relevance, and quality of AI system reasoning. My responsibilities included structured fact-checking and data validation to ensure the reliability of LLM outputs. I provided evidence-based analytical justifications and supported quality assurance in AI training workflows. • Conducted side-by-side (SxS) comparisons of LLM outputs for accuracy, safety, and logic. • Evaluated AI-generated content using objective rubrics and standards. • Provided structured feedback for continuous model improvement processes. • Utilized various digital platforms and browser tools for output documentation.

2025

AI Safety Policy Analyst (English) – LLM Red Teaming & RLHF

Redbrick AITextRLHF

In my role as an AI Safety Policy Analyst, I conducted structured LLM safety evaluations according to AI policy and trust & safety guidelines. I developed adversarial prompts (red teaming) to uncover vulnerabilities, bias, and harmful outputs in AI models. I also performed SxS response analysis and ranking for safety, quality, and reasoning evaluation. • Designed adversarial prompts and performed safety analysis tasks. • Conducted stack-rankings and side-by-side comparisons to assess model behavior. • Authored clear and defensible analytical rationales for evaluation results. • Contributed to alignment and reinforcement learning from human feedback (RLHF) processes for AI improvement.

In my role as an AI Safety Policy Analyst, I conducted structured LLM safety evaluations according to AI policy and trust & safety guidelines. I developed adversarial prompts (red teaming) to uncover vulnerabilities, bias, and harmful outputs in AI models. I also performed SxS response analysis and ranking for safety, quality, and reasoning evaluation. • Designed adversarial prompts and performed safety analysis tasks. • Conducted stack-rankings and side-by-side comparisons to assess model behavior. • Authored clear and defensible analytical rationales for evaluation results. • Contributed to alignment and reinforcement learning from human feedback (RLHF) processes for AI improvement.

2025 - 2025

Education

Work History