Rubricon: Automated Rubric Generation and Refinement for LLM Evaluation and Training
arXiv:2610.38220v1 Announce Type: new Abstract: Rubrics enable reward assignment on tasks without verifiable answers, but are expensive to manually produce from scratch. We present Rubricon, a pipeline that generates discriminative rubrics without requiring human preference labels, rubric annotation, or generator-model training. Using any LLM, Rubricon samples multiple rubrics, selects the most discriminative candidate, adds criteria to capture overlooked requirements or response flaws, and pru…