Katryna Peart

Key Skills

Software

Mercor

Google Cloud Vertex AI

Other

Internal/Proprietary Tooling

Top Subject Matter

Safety Domain Expertise

Policy Classification

Relevance Domain Expertise

Top Data Types

Text

Document

Image

Top Task Types

Evaluation Rating

Prompt Response Writing SFT

Red Teaming

Fine Tuning

Text Summarization

Question Answering

Text Generation

RLHF

Freelancer Overview

AI Content Strategist & Relevance Evaluator (Contract). Brings 10+ years of professional experience across legal operations, contract review, compliance, and structured analysis. Core strengths include Mercor, Uber AI Solutions, and Google Cloud Vertex AI. Education includes Master of Arts, Royal Holloway, University of London and Bachelor of Arts, New York University. AI-training focus includes data types such as Text and labeling workflows including Evaluation and Rating.

Labeling Experience

Independent AI Researcher & Model Evaluator

OtherText

This independent research and model evaluator role included designing and executing a tiered protocol for evaluating the output of multiple large language models. The work analyzed model certainty, authority, hedging, framing, and sensitivity in contested or ambiguous narratives. Red-teaming, rubric development, and domain-specific knowledge were used to assess accuracy, nuance, and risk. • Crafted structured protocols for systematic model testing. • Conducted detailed rubric-based evaluation on sensitive topics. • Applied practitioner knowledge in cultural and historical contexts. • Explored patterns in LLM behavior via prompt engineering and analysis.

This independent research and model evaluator role included designing and executing a tiered protocol for evaluating the output of multiple large language models. The work analyzed model certainty, authority, hedging, framing, and sensitivity in contested or ambiguous narratives. Red-teaming, rubric development, and domain-specific knowledge were used to assess accuracy, nuance, and risk. • Crafted structured protocols for systematic model testing. • Conducted detailed rubric-based evaluation on sensitive topics. • Applied practitioner knowledge in cultural and historical contexts. • Explored patterns in LLM behavior via prompt engineering and analysis.

2025 - Present

AI Content Strategist & Relevance Evaluator (Contract)

MercorText

This role involved evaluating high volumes of AI-generated text content for safety, policy alignment, and quality using structured rubrics. Outputs were classified into risk categories and supported with detailed evidence-based justifications. Systematic prompt-response evaluation was used to contribute annotated data for LLM training. • Evaluated outputs for hate, violence, sexual content, and self-harm. • Maintained high consistency through detailed guidelines and QA. • Provided concise, evidence-based explanations for decisions. • Enhanced dataset quality via structured annotation protocols.

This role involved evaluating high volumes of AI-generated text content for safety, policy alignment, and quality using structured rubrics. Outputs were classified into risk categories and supported with detailed evidence-based justifications. Systematic prompt-response evaluation was used to contribute annotated data for LLM training. • Evaluated outputs for hate, violence, sexual content, and self-harm. • Maintained high consistency through detailed guidelines and QA. • Provided concise, evidence-based explanations for decisions. • Enhanced dataset quality via structured annotation protocols.

2025 - 2026

Content Strategist / Editorial QA (Contract)

Google Cloud Vertex AIText

This contract role involved detailed editorial and safety QA on AI-generated textual content for Google products. Evaluations targeted policy compliance, risk categorization, and rubrics-based overall quality checks. High-volume annotations were performed with rigorous consistency and accuracy standards. • Evaluated outputs for policy and safety risks. • Provided actionable feedback to improve data quality. • Identified and flagged knowledge gaps for team remediation. • Delivered fast, accurate editorial reviews in high-velocity settings.

This contract role involved detailed editorial and safety QA on AI-generated textual content for Google products. Evaluations targeted policy compliance, risk categorization, and rubrics-based overall quality checks. High-volume annotations were performed with rigorous consistency and accuracy standards. • Evaluated outputs for policy and safety risks. • Provided actionable feedback to improve data quality. • Identified and flagged knowledge gaps for team remediation. • Delivered fast, accurate editorial reviews in high-velocity settings.

2025 - 2025

AI Prompt & Content Evaluator (Contract)

Text

This position consisted of reviewing and rating AI-generated communications and marketing text for safety and relevance according to established policy guidelines. Severity ratings were assigned and clear written justifications accompanied every label. Work contributed to model improvement and reliable dataset construction. • Performed systematic relevance and safety assessments of generated responses. • Collaborated on QA processes for rubric adherence. • Documented model errors and provided improvement feedback. • Supported consistent application of dataset labels across projects.

This position consisted of reviewing and rating AI-generated communications and marketing text for safety and relevance according to established policy guidelines. Severity ratings were assigned and clear written justifications accompanied every label. Work contributed to model improvement and reliable dataset construction. • Performed systematic relevance and safety assessments of generated responses. • Collaborated on QA processes for rubric adherence. • Documented model errors and provided improvement feedback. • Supported consistent application of dataset labels across projects.

2025 - 2025

Education

Work History