Joseph Fredrickson

Key Skills

Software

Data Annotation Tech

Labelbox

Remotasks

Scale AI

Internal/Proprietary Tooling

Top Subject Matter

No subject matter listed

Top Data Types

Computer Code Programming

Document

Text

Top Task Types

Evaluation Rating

Prompt Response Writing SFT

RLHF

Text Summarization

Freelancer Overview

I specialize in high-level LLM evaluation, prompt engineering, and AI content quality analysis. With a strong foundation in both creative and technical editing—having worked with bestselling authors and trained under Brandon Sanderson—I bring precision, clarity, and editorial insight to every AI output I assess. My background in mechanical engineering, neuromorphic computing, and programming also enables me to evaluate technical and STEM-related prompts with analytical rigor. I’ve contributed to projects focused on evaluating model helpfulness, factuality, coherence, and ethical alignment. From instructional prompts to complex edge case testing, I apply a multidisciplinary lens to refine AI behavior and improve large language model performance. With over a decade of experience in content editing, technical writing, and SEO strategy, I’m well-equipped to deliver gold-standard AI training data across multiple domains.

Labeling Experience

Alignerr

LabelboxImageBounding BoxClassification

Label various animals according to a bounding box as given via classifications.

Label various animals according to a bounding box as given via classifications.

2024

LLM Evaluation Specialist & Prompt Quality Rater – Outlier.ai

Scale AITextRLHF

Evaluated AI-generated responses for clarity, accuracy, logical coherence, tone, and helpfulness across thousands of prompt types, including instruction-following, ethical reasoning, conversational, and narrative tasks. Graded model outputs using rubric-based scoring systems and provided qualitative feedback for model refinement. Designed original prompts to test edge cases and improve behavior alignment in reinforcement learning pipelines. Reviewed and curated gold-standard data for supervised fine-tuning and model evaluation.

Evaluated AI-generated responses for clarity, accuracy, logical coherence, tone, and helpfulness across thousands of prompt types, including instruction-following, ethical reasoning, conversational, and narrative tasks. Graded model outputs using rubric-based scoring systems and provided qualitative feedback for model refinement. Designed original prompts to test edge cases and improve behavior alignment in reinforcement learning pipelines. Reviewed and curated gold-standard data for supervised fine-tuning and model evaluation.

2024

Creative Writing Prompt Engineer & Evaluator – Writ-OR.com

Internal Proprietary ToolingTextText GenerationEmotion Recognition

Developed structured prompts and evaluation criteria for an AI-powered story development platform supporting authors with genre-based outlines, character arcs, and scene progression. Created modular content pathways based on “Save the Cat Writes a Novel” and other narrative frameworks. Evaluated LLM-generated story beats, dialogue suggestions, and genre matches for accuracy, originality, and user clarity. Refined AI output through systematic review and rubric scoring, improving the assistant’s ability to respond like an editor or writing coach.

Developed structured prompts and evaluation criteria for an AI-powered story development platform supporting authors with genre-based outlines, character arcs, and scene progression. Created modular content pathways based on “Save the Cat Writes a Novel” and other narrative frameworks. Evaluated LLM-generated story beats, dialogue suggestions, and genre matches for accuracy, originality, and user clarity. Refined AI output through systematic review and rubric scoring, improving the assistant’s ability to respond like an editor or writing coach.

2022

Education

Work History