Reviewer (cypher_evals_c | Consensus & Revamp)
Text
As a Reviewer for cypher_evals_c, I reviewed and calibrated tasks focused on instruction following versus truthfulness and preference ranking. I ensured task adherence to mixed-language guidance, localization, and proper format and length. The position required providing structured justifications and aligning evaluation standards across reviewer teams.
• Calibrated multi-dimensional evaluation tasks.
• Applied format, length, and localization criteria to ratings.
• Documented guidelines and edge-case rationales for consensus.
• Maintained batch consistency with evidence-based justifications.
As a Reviewer for cypher_evals_c, I reviewed and calibrated tasks focused on instruction following versus truthfulness and preference ranking. I ensured task adherence to mixed-language guidance, localization, and proper format and length. The position required providing structured justifications and aligning evaluation standards across reviewer teams. • Calibrated multi-dimensional evaluation tasks. • Applied format, length, and localization criteria to ratings. • Documented guidelines and edge-case rationales for consensus. • Maintained batch consistency with evidence-based justifications.
2024 - Present