Colm Mullen

Key Skills

Software

Scale AI

Top Subject Matter

No subject matter listed

Top Data Types

Text

Top Task Types

Classification

Evaluation Rating

Prompt Response Writing SFT

RLHF

Translation Localization

Freelancer Overview

In my role as an LLM evaluator and prompt writer, I have been actively engaged in crafting nuanced prompts and analyzing AI-generated responses across key dimensions such as truthfulness, localization, and writing quality. My work in Reinforcement Learning from Human Feedback (RLHF) has required me to refine AI-generated outputs, identify areas for improvement, and provide targeted feedback to enhance model performance. This has deepened my understanding of how AI systems process and generate language, reinforcing my commitment to bridging the gap between human communication and machine learning. Additionally, my background as a translator and interpreter has given me a keen eye for linguistic precision, contextual accuracy, and cultural nuance. Having worked extensively in both English and Spanish, I bring a deep understanding of how language evolves across different registers and contexts. This expertise allows me to assess AI outputs with a critical lens, ensuring that language models align not just with grammatical correctness but also with regional and sociolinguistic expectations.

Labeling Experience

RLHF prompt writer, response evaluator, and rewriter

Scale AITextRLHF

In this RLHF LLM training project, it is my responsibility to create prompts that belong to a specific category (business, health, natural sciences, etc.) with enough natural constraints that at least one of the two responses generated contain a major or minor issue across specific dimensions. The dimensions which must be taken into consideration and labelled as Major issue, minor issue or no issue are Localization (in this case Spain), Instruction following, truthfulness, verbosity, writing quality, and harmlessness. After rating the responses across these dimensions, I must choose which of the two are better, according to their individual ratings. If both achieve the same rating, then I must rewrite the one I believe is better based on the weight of the issues contained in each.

In this RLHF LLM training project, it is my responsibility to create prompts that belong to a specific category (business, health, natural sciences, etc.) with enough natural constraints that at least one of the two responses generated contain a major or minor issue across specific dimensions. The dimensions which must be taken into consideration and labelled as Major issue, minor issue or no issue are Localization (in this case Spain), Instruction following, truthfulness, verbosity, writing quality, and harmlessness. After rating the responses across these dimensions, I must choose which of the two are better, according to their individual ratings. If both achieve the same rating, then I must rewrite the one I believe is better based on the weight of the issues contained in each.

2024

Education

Work History