For employers

Hire this AI Trainer

Sign in or create an account to invite AI Trainers to your job.

LoginCreate account
Invite to Job
J
John Kearney

John Kearney

AI Red Teamer & Eval Engineer - RLHF Specialist

USA flagChicago, Usa
$75.00/hrIntermediateAws SagemakerGoogle Cloud Vertex AIMercor

Key Skills

Software

AWS SageMakerAWS SageMaker
Google Cloud Vertex AIGoogle Cloud Vertex AI
MercorMercor
MindriftMindrift
Scale AIScale AI
Internal/Proprietary Tooling

Top Subject Matter

AI system adversarial robustness and vulnerability analysis
AI model safety evaluation
Adversarial datasets

Top Data Types

TextText
DocumentDocument
AudioAudio

Top Task Types

Red TeamingRed Teaming
Evaluation/RatingEvaluation/Rating
RLHFRLHF
Prompt + Response Writing (SFT)Prompt + Response Writing (SFT)
Fine-tuningFine-tuning
ClassificationClassification

Freelancer Overview

AI Red Teamer & Eval Engineer — AI Control Hackathon (Apart Research). Brings 2+ years of professional experience across complex professional workflows, research, and quality-focused execution. Core strengths include Internal and Proprietary Tooling. Education includes Bachelor of Science, University of Minnesota Twin Cities (2023) and Graduate-Level Preparation, Self-Directed Graduate-Level Preparation (2025). AI-training focus includes data types such as Text and labeling workflows including Red Teaming, Evaluation, and Rating.

IntermediateEnglishSpanishFrench

Labeling Experience

Evaluator & Vulnerability Auditor — 15 Research Lab

Text
Audited, labeled, and verified vulnerabilities in major AI evaluation frameworks by conducting structured safety and security testing. Identified issues in judge and classifier prompts and contributed confirmed findings to open issues and pull requests for remediation. Documented types of prompt injection and adversarial behaviors in frameworks such as ControlArena and HarmBench. • Executed targeted prompt injection labeling and evaluation on leading frameworks. • Provided structured vulnerability reports with detailed labeling of adversarial events. • Maintained traceable logs of attack attempts and framework responses. • Supported external confirmation and remediation of labeling-based findings.

Audited, labeled, and verified vulnerabilities in major AI evaluation frameworks by conducting structured safety and security testing. Identified issues in judge and classifier prompts and contributed confirmed findings to open issues and pull requests for remediation. Documented types of prompt injection and adversarial behaviors in frameworks such as ControlArena and HarmBench. • Executed targeted prompt injection labeling and evaluation on leading frameworks. • Provided structured vulnerability reports with detailed labeling of adversarial events. • Maintained traceable logs of attack attempts and framework responses. • Supported external confirmation and remediation of labeling-based findings.

2026 - Present

Dataset & Benchmark Creator — 15 Research Lab

Text
Created and curated evaluation datasets and benchmarks designed to measure AI safety, adversarial resilience, and model compliance under various threat scenarios. Labeled and scored structured adversarial scenarios as well as created prompt injection test cases and payload lists for AI models. Mapped taxonomy and benchmarked agent safety compliance across multiple agents and frameworks. • Developed over 10,000 adversarial scenario labels for deployment failure analysis. • Composed scored test sets across multiple threat categories for model assessment. • Constructed AI SecLists with categorized payloads and encoding/mutation types. • Produced compliance leaderboards and safety benchmarks for agent tool use.

Created and curated evaluation datasets and benchmarks designed to measure AI safety, adversarial resilience, and model compliance under various threat scenarios. Labeled and scored structured adversarial scenarios as well as created prompt injection test cases and payload lists for AI models. Mapped taxonomy and benchmarked agent safety compliance across multiple agents and frameworks. • Developed over 10,000 adversarial scenario labels for deployment failure analysis. • Composed scored test sets across multiple threat categories for model assessment. • Constructed AI SecLists with categorized payloads and encoding/mutation types. • Produced compliance leaderboards and safety benchmarks for agent tool use.

2026 - Present

AI Red Teamer & Eval Engineer — AI Control Hackathon (Apart Research)

TextRed Teaming
Participated in an AI Control Hackathon focusing on evaluating the robustness and vulnerabilities of large language models via adversarial interactions. Generated and analyzed attack transcripts and conducted over a hundred model evaluations for systems such as Opus 4.6 and Sonnet 4.5. Developed proof-of-concept monitoring approaches to enhance detection and failure analysis for text-based adversarial scenarios. • Evaluated and red-teamed LLM deployment with iterative attack/response protocols. • Simulated advanced attack classes including prompt injection and salami attacks. • Produced detailed reports on monitor failures and model evasion strategies. • Contributed to benchmark scenarios for measuring AI safety and alignment.

Participated in an AI Control Hackathon focusing on evaluating the robustness and vulnerabilities of large language models via adversarial interactions. Generated and analyzed attack transcripts and conducted over a hundred model evaluations for systems such as Opus 4.6 and Sonnet 4.5. Developed proof-of-concept monitoring approaches to enhance detection and failure analysis for text-based adversarial scenarios. • Evaluated and red-teamed LLM deployment with iterative attack/response protocols. • Simulated advanced attack classes including prompt injection and salami attacks. • Produced detailed reports on monitor failures and model evasion strategies. • Contributed to benchmark scenarios for measuring AI safety and alignment.

2026 - 2026

Education

U

University of Minnesota Twin Cities

Bachelor of Science, General Science

Bachelor of Science
2020 - 2023
S

Self-Directed Graduate-Level Preparation

Graduate-Level Preparation, Machine Learning and Artificial Intelligence Safety

Graduate-Level Preparation
2025

Work History

A

Authensor

Founder and Lead Engineer

Chicago
2025 - Present
1

15 Research Lab

Founder and Lead Researcher

Chicago
2025 - Present