Annotation Reviewer
RLHF annotation and review for a large language model alignment project. Promoted from Annotator to Reviewer within 2 weeks based on annotation quality and throughput. Scope: Evaluate pairs of LLM-generated responses against user prompts using a structured rubric covering instruction following, completeness, factual accuracy, coherence, relevance, efficiency, articulation, and length appropriateness. Each task involves writing detailed Strengths and Areas of Improvement with cited excerpts, assigning 1-5 quality scores per response, and determining preference rankings with justification. As Reviewer: Independently complete full annotations, then audit other annotators' work — approving, providing corrective feedback, or flagging substantial misses in Strengths/AOIs assessments. Quality measures: Fact-checking all factual claims against primary sources, cross-referencing annotations against a 15-section internal style guide, and running systematic QC checklists before submission. Bilingual coverage (English and Simplified Chinese).