Decision and System One models in rheumatology: accuracy and calibration on specialty-training examination questions and diagnostic case vignettes
arXiv:2610.40318v1 Announce Type: cross Abstract: Objectives: To evaluate the accuracy and calibration of Decision and System One models on rheumatology examination questions and diagnostic vignettes. Methods: Two benchmarks were analysed: 172 rheumatology questions from Spanish specialty-training examinations (expanded RheumaMIR dataset) in Spanish and English, and 70 English diagnostic vignettes from RheumBench with 5-18 candidate diagnoses, 52 of which included the reference diagnosis. Four…