Senior Python Engineer
* The trainer job involves writing a prompt according to the task's metadata and evaluating the written prompt and the model responses following their criteria on "Instruction Following (If the model does all the prompt asks for)", "Accuracy (if the code has errors or does what is asked for correctly)", "Efficiency (if the code does the task in the best manner or could be implemented better)", "Presentation (If either the code needs better comments or the markdown presentation has issues)", "Up-to-date (if the code uses deprecated libraries or functions)", "Executable Code (If the given code is executable as is by copying it to a file and ran or how much of the code was given by the model)", and "Other issues (any issue in the response that does not belong to the other categories.) * The pod lead does some training workbooks, review workbooks done by trainers. * The calibrator evaluates the reviews from leads.