Costello Joseph

Key Skills

Software

Label Studio

Labelbox

Top Subject Matter

Large language models

generative AI

Rlhf Domain Expertise

Top Data Types

Text

Image

Top Task Types

RLHF

Object Detection

Fine-tuning

Entity (NER) Classification

Classification

Freelancer Overview

Senior AI Data Annotator & Labeling Lead, OpenAI. Brings 17+ years of professional experience across complex professional workflows, research, and quality-focused execution. Core strengths include Label Studio and Labelbox. Education includes Doctor of Philosophy, Stanford University (2014) and Master of Science, MIT (2010). AI-training focus includes data types such as Text and Image and labeling workflows including RLHF, Object Detection, and Fine-tuning.

Labeling Experience

Generative AI Content Annotation Project

Label StudioImage

I labeled and reviewed generative AI content outputs from diffusion models for image-text alignment, prompt adherence, and quality scoring. My work ensured high-fidelity, on-task image generations for model development. I contributed insights to cross-modal annotation standards. • Performed evaluation and scoring of AI-generated images • Assessed text-image alignment accuracy and prompt following • Provided feedback on model output quality • Used Label Studio and Scale AI for project execution

I labeled and reviewed generative AI content outputs from diffusion models for image-text alignment, prompt adherence, and quality scoring. My work ensured high-fidelity, on-task image generations for model development. I contributed insights to cross-modal annotation standards. • Performed evaluation and scoring of AI-generated images • Assessed text-image alignment accuracy and prompt following • Provided feedback on model output quality • Used Label Studio and Scale AI for project execution

2022 - Present

Senior AI Data Annotator & Labeling Lead, OpenAI

Label StudioTextRLHF

As Senior AI Data Annotator & Labeling Lead at OpenAI, I curated and annotated large-scale NLP datasets for training and fine-tuning state-of-the-art language models. I led a team overseeing multi-type data labeling projects and personally performed RLHF evaluations on model outputs. I helped develop annotation guidelines and maintain inter-annotator consistency. • Managed quality assurance for millions of text, image, and multimodal annotations • Conducted RLHF (Reinforcement Learning from Human Feedback) ranking and scoring tasks • Collaborated on prompt engineering and guideline updates • Used Label Studio, Labelbox, and Scale AI extensively across teams

As Senior AI Data Annotator & Labeling Lead at OpenAI, I curated and annotated large-scale NLP datasets for training and fine-tuning state-of-the-art language models. I led a team overseeing multi-type data labeling projects and personally performed RLHF evaluations on model outputs. I helped develop annotation guidelines and maintain inter-annotator consistency. • Managed quality assurance for millions of text, image, and multimodal annotations • Conducted RLHF (Reinforcement Learning from Human Feedback) ranking and scoring tasks • Collaborated on prompt engineering and guideline updates • Used Label Studio, Labelbox, and Scale AI extensively across teams

2018 - Present

Bias Mitigation Dataset Curation

LabelboxTextClassification

During the Bias Mitigation Dataset Curation project, I annotated training corpora to flag, correct, and document biased content. I worked to improve equity and reduce toxic outputs in downstream AI models. My efforts supported the development of fairer and safer AI systems. • Applied strategies for bias identification and remediation • Maintained documentation of labeling rationale • Collaborated with model developers for bias audits • Used Labelbox and internal tools for annotation

During the Bias Mitigation Dataset Curation project, I annotated training corpora to flag, correct, and document biased content. I worked to improve equity and reduce toxic outputs in downstream AI models. My efforts supported the development of fairer and safer AI systems. • Applied strategies for bias identification and remediation • Maintained documentation of labeling rationale • Collaborated with model developers for bias audits • Used Labelbox and internal tools for annotation

2016 - 2018

AI Data Labeler & Annotation Specialist, Google DeepMind

LabelboxImageObject Detection

At Google DeepMind, I annotated computer vision datasets for object detection and image classification, enhancing model precision through accurate labeling. I employed prompt-guided annotation strategies and audited data for bias. I mentored team members on best practices in annotation. • Labeled images and cross-modal datasets for vision and NLP • Removed biased and low-quality labels for fairer models • Led inter-annotator agreement reviews • Used Labelbox and Scale AI for annotation pipelines

At Google DeepMind, I annotated computer vision datasets for object detection and image classification, enhancing model precision through accurate labeling. I employed prompt-guided annotation strategies and audited data for bias. I mentored team members on best practices in annotation. • Labeled images and cross-modal datasets for vision and NLP • Removed biased and low-quality labels for fairer models • Led inter-annotator agreement reviews • Used Labelbox and Scale AI for annotation pipelines

2014 - 2018

Data Annotation & ML Data Specialist, Microsoft Research

Label StudioTextFine Tuning

While at Microsoft Research, I processed and labeled diverse large-scale text and image data for live ML training pipelines. I applied prompt engineering and established best practices for high-quality annotation. I contributed to open-source annotation tools and structured workflows. • Labeled text and image datasets for real-time ML models • Developed prompt engineering workflows for label quality • Improved open-source annotation tools • Used Label Studio and custom internal platforms

While at Microsoft Research, I processed and labeled diverse large-scale text and image data for live ML training pipelines. I applied prompt engineering and established best practices for high-quality annotation. I contributed to open-source annotation tools and structured workflows. • Labeled text and image datasets for real-time ML models • Developed prompt engineering workflows for label quality • Improved open-source annotation tools • Used Label Studio and custom internal platforms

2012 - 2014

Show more

Education

Work History