Betty Tai

Key Skills

Software

Scale AI

Mercor

Labelbox

Top Subject Matter

AI Model Evaluation and Data Engineering (STEM Domains)

Reinforcement Learning and LLM Training for SQL Agents

Weather Forecast Annotation and Model Evaluation

Top Data Types

Text

Image

Document

Top Task Types

Computer Programming/Coding

Evaluation/Rating

RLHF

Function Calling

Data Collection

Fine-tuning

Red Teaming

Prompt + Response Writing (SFT)

Freelancer Overview

STEM Fellow — Human Frontier Collective | Scale AI. Brings 11+ years of professional experience across legal operations, contract review, compliance, and structured analysis. Core strengths include Scale AI, Internal, and Proprietary Tooling. Education includes Doctor of Philosophy, N/A (2025) and Master of Science, Johns Hopkins University (2020). AI-training focus includes data types such as Text, Computer Code, and Programming and labeling workflows including Evaluation, Rating, and Data Collection.

Labeling Experience

Principal Consultant | BearResearch Labs, LLC.

Data Collection

Engineered reinforcement learning training infrastructure for text-to-SQL data agents, creating multi-domain enterprise databases. Curated BIRD-style question-SQL pairs, schema documentation, and data dictionaries for robust training. Developed validation pipelines and metadata-enriched data to support data-driven RL policy development. Designed complexity-based curriculum sampling for advanced agent training. • Architected 13 enterprise-domain databases with structured training data for text-to-SQL tasks. • Developed RL environment tools and validation pipelines for agentic systems. • Curated datasets with gold SQL actions, chain-of-thought evidence, and structured metadata. • Implemented curriculum sampling for complex policy conditioning in RL agents.

Engineered reinforcement learning training infrastructure for text-to-SQL data agents, creating multi-domain enterprise databases. Curated BIRD-style question-SQL pairs, schema documentation, and data dictionaries for robust training. Developed validation pipelines and metadata-enriched data to support data-driven RL policy development. Designed complexity-based curriculum sampling for advanced agent training. • Architected 13 enterprise-domain databases with structured training data for text-to-SQL tasks. • Developed RL environment tools and validation pipelines for agentic systems. • Curated datasets with gold SQL actions, chain-of-thought evidence, and structured metadata. • Implemented curriculum sampling for complex policy conditioning in RL agents.

2025 - Present

STEM Fellow — Human Frontier Collective | Scale AI

Scale AIText

Designed and engineered evaluation and training datasets to assess and fine-tune frontier AI models for STEM domains. Applied systematic evaluation methodologies and adversarial testing techniques to ensure robust model performance. Provided expert-level validation on AI system outputs to improve data quality and strategy. Collaborated with PhD-level researchers to publish applied AI evaluation benchmarks. • Developed structured data artifacts and evaluation benchmarks for multiple STEM-related domains. • Applied adversarial and systematic evaluation methodologies for dataset quality assurance. • Informed AI training data and safe-output validation strategies. • Published research advancing model robustness in collaboration with subject matter experts.

Designed and engineered evaluation and training datasets to assess and fine-tune frontier AI models for STEM domains. Applied systematic evaluation methodologies and adversarial testing techniques to ensure robust model performance. Provided expert-level validation on AI system outputs to improve data quality and strategy. Collaborated with PhD-level researchers to publish applied AI evaluation benchmarks. • Developed structured data artifacts and evaluation benchmarks for multiple STEM-related domains. • Applied adversarial and systematic evaluation methodologies for dataset quality assurance. • Informed AI training data and safe-output validation strategies. • Published research advancing model robustness in collaboration with subject matter experts.

2025 - Present

Technology Architect (R&D) | NOAA - National Weather Service HQ

Text

Developed automated tools and pipelines for auditing and managing large-scale annotated datasets from multi-lingual AI data annotators. Ensured high throughput and accurate labeling to meet evaluation standards for weather AI systems. Piloted GenAI natural language processing products to improve forecast translation quality. Utilized prompt engineering to enhance human-readable model outputs. • Validated high-volume evaluation datasets for accuracy and consistency. • Built pipelines for managing multi-lingual annotation data and throughput. • Applied prompt engineering for natural language accessibility in weather forecasts. • Advanced automated annotation auditing techniques for quality assurance.

Developed automated tools and pipelines for auditing and managing large-scale annotated datasets from multi-lingual AI data annotators. Ensured high throughput and accurate labeling to meet evaluation standards for weather AI systems. Piloted GenAI natural language processing products to improve forecast translation quality. Utilized prompt engineering to enhance human-readable model outputs. • Validated high-volume evaluation datasets for accuracy and consistency. • Built pipelines for managing multi-lingual annotation data and throughput. • Applied prompt engineering for natural language accessibility in weather forecasts. • Advanced automated annotation auditing techniques for quality assurance.

2023 - 2024

Education

Work History