For employers

Hire this AI Trainer

Sign in or create an account to invite AI Trainers to your job.

LoginCreate account
Invite to Job
O

Olusegun Oyemade Makanju

Human-in-the-loop Evaluator & LLM API Developer

Canada flagDurham, Canada
Expert

Key Skills

Software

No software listed

Top Subject Matter

LLM Evaluation / AI Model Outputs
Prompt Engineering / LLM Classifier Fine-tuning
Legal Services & Contract Review

Top Data Types

TextText
DocumentDocument

Top Task Types

Prompt + Response Writing (SFT)Prompt + Response Writing (SFT)

Freelancer Overview

Human-in-the-loop Evaluator & LLM API Developer. Brings 7+ years of professional experience across complex professional workflows, research, and quality-focused execution. Core strengths include Internal and Proprietary Tooling. Education includes Bachelor of Science, Carleton University. AI-training focus includes data types such as Text and labeling workflows including Evaluation, Rating, and Prompt + Response Writing (SFT).

Expert

Labeling Experience

Prompt Engineer & LLM Response Rater

TextPrompt Response Writing SFT
I executed few-shot prompting strategies and supervised prompt and response writing tasks to fine-tune LLM classifiers. My responsibilities included curating example datasets, writing prompts, generating and rating LLM outputs, and maintaining prompt/response quality for sub-300ms API endpoints. This experience covered prompt engineering, output review, and structured fast feedback to enhance classifier accuracy. • Designed and tested prompts for LLM classifiers using few-shot learning techniques. • Rated LLM-generated outputs for relevance, accuracy, and coherence. • Maintained quality standards for prompt/response datasets. • Supported rapid inference and continuous LLM evaluation with custom metrics.

I executed few-shot prompting strategies and supervised prompt and response writing tasks to fine-tune LLM classifiers. My responsibilities included curating example datasets, writing prompts, generating and rating LLM outputs, and maintaining prompt/response quality for sub-300ms API endpoints. This experience covered prompt engineering, output review, and structured fast feedback to enhance classifier accuracy. • Designed and tested prompts for LLM classifiers using few-shot learning techniques. • Rated LLM-generated outputs for relevance, accuracy, and coherence. • Maintained quality standards for prompt/response datasets. • Supported rapid inference and continuous LLM evaluation with custom metrics.

2023 - Present

Human-in-the-loop Evaluator & LLM API Developer

Text
I contributed to human-in-the-loop evaluation and confidence scoring for LLM systems as part of end-to-end project deployments. My work involved building and deploying APIs that enabled uncertainty-based automated human review, helping calibrate LLM outputs and triggering manual checks when low confidence was detected. I also implemented evaluation metrics endpoints and integrated flagging systems for AI model outputs. • Developed, deployed, and monitored LLM system APIs with HITL triggers. • Applied Bayesian uncertainty quantification for automated human review workflows. • Labeling tasks focused on evaluation, confidence scoring, and manual rating of LLM-generated text. • Used internal/proprietary tooling with FastAPI, Groq, and Docker for both automation and manual review.

I contributed to human-in-the-loop evaluation and confidence scoring for LLM systems as part of end-to-end project deployments. My work involved building and deploying APIs that enabled uncertainty-based automated human review, helping calibrate LLM outputs and triggering manual checks when low confidence was detected. I also implemented evaluation metrics endpoints and integrated flagging systems for AI model outputs. • Developed, deployed, and monitored LLM system APIs with HITL triggers. • Applied Bayesian uncertainty quantification for automated human review workflows. • Labeling tasks focused on evaluation, confidence scoring, and manual rating of LLM-generated text. • Used internal/proprietary tooling with FastAPI, Groq, and Docker for both automation and manual review.

2022 - Present

Education

C

Carleton University

Bachelor of Science, Data Science

Bachelor of Science
Not specified

Work History

T

Thermo Fisher Scientific

Data Automation and Customer Solutions Specialist

Durham
2022 - Present
C

Canadian Tire

Distribution Operations Specialist

N/A
2021 - 2022