Why I Ditched "Just Let the LLM Handle It" for a State Machine (And Slept Better at Night)
When I started building my AI technical interviewer, I did what most people do: I threw a big system prompt at the LLM and told it to "act like an interviewer, ask coding questions, give hints when the candidate is stuck, and score them at the end." It worked... about 80% of the time. The other 20% is what this post is about. Here's what kept happening. Mid-interview, the model would: Randomly decide the interview was "done" after one question Give away the answer while trying to give a "gentle…