Run by OpenTrain for a client. OpenTrain runs this project for a client company. The client's name is confidential.
OpenTrain reviews your application, your CV and your AI interview. OpenTrain can send your name, email address, country, rate, CV and interview answers to the client company for this project, so that the client can review your application. The client may contact you about next steps.
Data Type
Task Types
Subject Matter / Industry
Language
Service Model
Share link
5 labels per item
500 labels
We’re looking for an analytical scenario writer with strong QA-style thinking and excellent written English. You should be comfortable designing structured evaluation scenarios, defining expected (“gold standard”) agent behavior, and working with structured formats like JSON/YAML. A background in software testing, QA, data analysis, or NLP annotation is strongly preferred. Basic Python and JavaScript experience is required. What you’ll be doing: You’ll design realistic, reusable evaluation scenarios for LLM-based agents that simulate real-world tasks. You’ll define the golden path and acceptable behaviors, annotate task steps and expected outputs, and document edge cases and scoring logic. You’ll also review agent outputs, iterate on scenarios for clarity and coverage, and collaborate with developers and other contributors to test and refine evaluation frameworks.