Claudia Helming

Key Skills

Software

Appen

CrowdSource

Labelbox

OneForma

Remotasks

Scale AI

Toloka

Internal/Proprietary Tooling

Top Subject Matter

No subject matter listed

Top Data Types

Audio

Document

Text

Top Task Types

Evaluation Rating

Fine Tuning

Prompt Response Writing SFT

Red Teaming

RLHF

Freelancer Overview

I am an experienced AI contributor and reviewer with a proven track record across a wide spectrum of training data projects for large language models and multimodal AI systems. My work has spanned not only RLHF, red teaming, reasoning evaluation, STEM annotation, and prompt/response writing, but also included complex tasks involving both text, image, and video data—covering annotation, evaluation, and quality assurance in English and German. As a member of Outlier.ai’s Oracle Club, I was consistently ranked among the top contributors and entrusted with reviewer and mentoring responsibilities across diverse, often highly specialized projects. My responsibilities ranged from evaluating nuanced reasoning chains and ethical judgments to designing and reviewing prompts, as well as annotating and assessing multimodal datasets, ensuring high-quality outputs for next-generation AI models. My academic background in literature and linguistics and two decades in the tech industry as a founder and product strategist enable me to bring a unique, user-centric perspective to every task.

Labeling Experience

Multimodal Live Interaction Assessment & RLHF Evaluation for Conversational AI

Scale AIAudioRLHF

As a senior contributor and reviewer, I participated in the evaluation of advanced conversational AI systems’ new live interaction modes. This project involved designing and executing comprehensive test scenarios to assess live conversation capabilities, memory functions, and real-time user interactions across multiple modalities, including audio and video data. My responsibilities included: Multimodal annotation and quality assurance (text, image, video, live interaction) Creating test scenarios, conducting live conversations, providing structured feedback to improve model performance and evaluation guidelines, applying detailed RLHF-based rating scales for model output evaluation, and reviewing, rating, and optimizing other contributors' tasks. The project contributed directly to the refinement and quality assurance of next-generation AI systems for leading industry clients.

As a senior contributor and reviewer, I participated in the evaluation of advanced conversational AI systems’ new live interaction modes. This project involved designing and executing comprehensive test scenarios to assess live conversation capabilities, memory functions, and real-time user interactions across multiple modalities, including audio and video data. My responsibilities included: Multimodal annotation and quality assurance (text, image, video, live interaction) Creating test scenarios, conducting live conversations, providing structured feedback to improve model performance and evaluation guidelines, applying detailed RLHF-based rating scales for model output evaluation, and reviewing, rating, and optimizing other contributors' tasks. The project contributed directly to the refinement and quality assurance of next-generation AI systems for leading industry clients.

2024

Education

Work History