LLMs as Judges and Examinees for Medical Item Difficulty Prediction
arXiv:2610.38402v1 Announce Type: cross Abstract: Predicting medical item difficulty from item text alone remains challenging. Prior work has largely relied on linguistic features or text representations derived from general-purpose and domain-specific language models, but predictive performance has remained modest. This study takes a different approach by using difficulty-related signals that emerge when open-weight large language models (LLMs) interact with test items. Using USMLE and PANCE m…