RHLF for LLMs
One of the key projects I worked on involved document-based datasets for AI fine-tuning through Reinforcement Learning with Human Feedback (RHLF). My role was to evaluate, rate, and rank AI-generated responses to text prompts so they could be used to train and improve large language models. The scope covered reviewing documents across different domains — from business and technical contexts to general knowledge — and assessing them on parameters like factual accuracy, coherence, clarity, tone, and safety. To ensure consistency, I followed detailed labelling guidelines that outlined how to score responses, handle ambiguous or borderline cases, and flag content that was harmful, biased, or irrelevant. In situations where guidelines didn’t fully address an edge case, I documented my reasoning to support improvements in the rules. Over the course of a year, I worked on several thousand responses, maintaining both speed and quality. This experience taught me how critical well-structured, h