For employers

Hire this AI Trainer

Sign in or create an account to invite AI Trainers to your job.

Invite to Job
Jay Dimmick

Jay Dimmick

AI Model Evaluation Contributor (Contract), OpenAI

USA flagAustin, Usa
$60.00/hrIntermediateCrowdsourceData Annotation TechGoogle Cloud Vertex AI

Key Skills

Software

CrowdSourceCrowdSource
Data Annotation TechData Annotation Tech
Google Cloud Vertex AIGoogle Cloud Vertex AI
Micro1
MercorMercor

Top Subject Matter

Insurance claims
Training & Development
Legal

Top Data Types

TextText
DocumentDocument
VideoVideo

Top Task Types

Object Detection
Text Generation
Question Answering
Text Summarization
RLHF
Fine Tuning
Transcription
Evaluation Rating
Data Collection
Prompt Response Writing SFT
Polygon

Freelancer Overview

AI Model Evaluation Contributor (Contract), OpenAI. Brings 16+ years of professional experience across legal operations, contract review, compliance, insurance and structured analysis. Core strengths include Internal and Proprietary Tooling. Education includes Bachelor of Science, University of Phoenix (2022). AI-training focus includes data types such as Text and labeling workflows including Evaluation and Rating.

IntermediateEnglish

Labeling Experience

AI Knowledge Assistant Creator & Evaluator, Allstate Insurance

Text
Built and evaluated a governed AI assistant (Leo) for decision support inside Allstate. Led accuracy checks and error correction on AI model outputs in claims and policy logic. Developed validation controls and human-in-the-loop workflows to ensure AI decisions matched expert understanding. • Identified and documented model hallucinations in insurance outputs. • Translated insurance workflows into structured training data for AI. • Piloted evaluation with 20 claims operations staff for iterative improvement. • Established rigorous feedback loops for claims and policy AI behavior.

Built and evaluated a governed AI assistant (Leo) for decision support inside Allstate. Led accuracy checks and error correction on AI model outputs in claims and policy logic. Developed validation controls and human-in-the-loop workflows to ensure AI decisions matched expert understanding. • Identified and documented model hallucinations in insurance outputs. • Translated insurance workflows into structured training data for AI. • Piloted evaluation with 20 claims operations staff for iterative improvement. • Established rigorous feedback loops for claims and policy AI behavior.

2025 - Present

AI Model Evaluation Contributor (Contract), OpenAI

Text
Conducted structured evaluation of large language model outputs for insurance and enterprise data. Assessed model reasoning quality, factual accuracy, and instruction adherence by analyzing generated text. Designed and rated insurance-specific prompts to test model performance in complex scenarios. • Evaluated LLM outputs using metacognitive rationale. • Documented model failures and inconsistencies for AI improvement. • Focused on insurance domain-specific data and enterprise contexts. • Supported AI training dataset integrity and quality.

Conducted structured evaluation of large language model outputs for insurance and enterprise data. Assessed model reasoning quality, factual accuracy, and instruction adherence by analyzing generated text. Designed and rated insurance-specific prompts to test model performance in complex scenarios. • Evaluated LLM outputs using metacognitive rationale. • Documented model failures and inconsistencies for AI improvement. • Focused on insurance domain-specific data and enterprise contexts. • Supported AI training dataset integrity and quality.

2025 - Present

Education

U

University of Phoenix

Bachelor of Science, Marketing

Bachelor of Science
2022 - 2022

Work History

A

Allstate Insurance

Senior Consultant II — Claims Operations & AI Enablement

Austin
2023 - Present
L

Liberty Mutual

Senior Claims Consultant — Complex Litigation & Risk Management

N/A
2022 - 2023