The fine-tune that looked fine
My fine-tuned AI model matched the original on the headline metric and cut false alarms by more than half. It also found 6 of the 14 roles that mattered. The original found 10. 0.707 vs 0.705 Rank correlation with Claude. A tie. 10 vs 22 False alarms. Better. 6 vs 10 of 14 Strong roles found. Worse. I fine-tuned a local AI model to judge job descriptions the way Claude does, then tested it on 116 roles it had never seen. On the averages, it looked like an upgrade. On the roles that mattered, it…