Jami Thomas

Key Skills

Software

Scale AI

Top Subject Matter

No subject matter listed

Top Data Types

Text

Top Label Types

Question Answering

Text Generation

Translation Localization

Freelancer Overview

I am an experienced AI Trainer and Model Evaluation Specialist with a strong background in data annotation, labeling, and human-in-the-loop evaluation for large language models. My expertise includes assessing AI-generated responses for accuracy, reasoning quality, and instruction adherence, as well as detecting errors, hallucinations, and inconsistencies. I have a proven track record of providing structured feedback to enhance model performance, conducting high-volume annotation tasks with consistent quality, and contributing to benchmarking exercises to ensure AI outputs meet rigorous standards. I am skilled in using annotation tools, AI evaluation platforms, and collaborative documentation systems, and I am passionate about improving the accuracy, safety, and reliability of AI systems through detailed analysis and quality assurance.

Labeling Experience

LLM Response Evaluation & RLHF Quality Assessment Project

Scale AITextQuestion AnsweringText Generation

Worked on a large-scale AI model evaluation project focused on improving large language model (LLM) performance through structured human feedback. Responsibilities included: Evaluated and rated 1,000+ AI-generated responses for factual accuracy, reasoning depth, coherence, safety compliance, and instruction adherence. Compared multiple model outputs and ranked them using structured evaluation rubrics. Identified hallucinations, logical inconsistencies, bias risks, and incomplete reasoning. Provided detailed written feedback to support reinforcement learning from human feedback (RLHF) pipelines. Flagged edge cases and ambiguous prompts to improve model robustness and alignment. Maintained 95%+ consistency across evaluation calibration benchmarks. Followed strict quality assurance standards including rubric adherence, consistency checks, and peer calibration reviews.

Worked on a large-scale AI model evaluation project focused on improving large language model (LLM) performance through structured human feedback. Responsibilities included: Evaluated and rated 1,000+ AI-generated responses for factual accuracy, reasoning depth, coherence, safety compliance, and instruction adherence. Compared multiple model outputs and ranked them using structured evaluation rubrics. Identified hallucinations, logical inconsistencies, bias risks, and incomplete reasoning. Provided detailed written feedback to support reinforcement learning from human feedback (RLHF) pipelines. Flagged edge cases and ambiguous prompts to improve model robustness and alignment. Maintained 95%+ consistency across evaluation calibration benchmarks. Followed strict quality assurance standards including rubric adherence, consistency checks, and peer calibration reviews.

2024

Education

Work History