A Jury, Not an Electorate: Label-Free Reliability Estimation for Ensembles of Language Models
arXiv:2610.21085v1 Announce Type: cross Abstract: An ensemble of language models is a jury, not an electorate. The answers are testimony from witnesses of unknown reliability, and the question is not what the majority said but who is credible. Most ensemble methods assign each expert a fixed reputation, often learned from labels. We ask whether it can instead be inferred from the pattern of agreement and disagreement among the experts. We model the experts as a jury: each question has a latent…