Prompt Engineer - Rubrics Design
As a basis, a user prompt is given, along with three generic responses to that prompt. After generating the generic responses, I select the best one and, if editing is necessary, I edit it as a golden response to the user prompt. I then classify the three responses as poor, good, or excellent. After classification, the process of creating atomic and exhaustive criteria for a perfect rubric begins so that the model understands what the user expects as a response to the prompt. Test several times if the criterion affects the response for better performance and if it is a self-contained and atomic criterion for each part of the prompt, if it follows the classification, for example: role-play, summary, paraphrase instructions to follow, list, etc. And if the objective/subjective, implicit/explicit criteria are followed, as well as classifying and detecting what type of intent the prompt vs. response has. After carefully reviewing and confirming that all rubric criteria are complete without