RLHF & Side-by-Side Preference Labeling (LLM Training Data)
Labeled and evaluated LLM outputs using rubric-based scoring and preference ranking (SxS) to generate high-quality RLHF training data. Tasks included rating helpfulness, correctness, reasoning quality, instruction adherence, and safety/format compliance; tagging error types (hallucination, missing constraints, math/logic issues, unsupported claims); and producing concise, consistent rationales aligned to guidelines. Maintained quality through calibration with benchmarks, spot-checking difficult edge cases, and applying consistent decision rules to reduce annotator drift.