为什么我放弃“全交给 LLM”，改用状态机

When I started building my AI technical interviewer, I did what most people do: I threw a big system prompt at the LLM and told it to "act like an interviewer, ask coding questions, give hints when the candidate is stuck, and score them at the end." It worked... about 80% of the time. The other 20% is what this post is about. The problem with trusting an LLM to run your whole flow Here's what kept happening. Mid-interview, the model would: Randomly decide the interview was "done" after one question Give away the answer while trying to give a "gentle hint" Forget it already asked a question and ask it again Jump straight to scoring before the candidate even submitted code None of this was a prompting skill issue. I rewrote that system prompt probably fifteen times. The real issue is structural: an LLM generating the next thing to say has no actual memory of "what phase are we in," unless you spoon-feed it that context perfectly, every single turn, forever. And even then, it can just... decide to do something else. It's a language model, not a state tracker. Treating it like one is where things fall apart. For a casual chatbot, that's fine — mild chaos is charming. For a product where someone's actual hiring decision depends on the interview going through every stage correctly, "mild chaos" is a support ticket and an angry candidate. What I did instead I pulled the structure of the interview out of the LLM entirely and put it into a plain old finite state machine. Something like: SETUP → GREETING → QUESTION_ASKED → CANDIDATE_CODING → HINT_CHECK → EVALUATING → SCORING → DONE Enter fullscreen mode Exit fullscreen mode The state machine — not the model — decides what phase we're in and what's allowed to happen next. The LLM only gets called inside a state, to do the one thing that state needs: generate a question, generate a hint, or generate a scorecard. It never gets to decide "we're done now" or "let's skip to scoring." That's not its job anymore. Practically, this meant: The LLM can't hallucinate a phase transition because it doesn't control phase transitions Hints only trigger off a real signal — 35+ seconds of no keystroke activity — not the model deciding "the candidate seems stuck" Scoring only fires after code is actually submitted and executed in the sandbox, not whenever the model feels like wrapping up The LLM still does all the hard, "actually intelligent" work — writing a good question, phrasing a helpful hint, writing a fair evaluation. It's just not allowed to drive the car anymore. It's a really good passenger with really good opinions. A simplified look at the transition logic class InterviewState(Enum): GREETING = "greeting" QUESTION_ASKED = "question_asked" CANDIDATE_CODING = "candidate_coding" HINT_CHECK = "hint_check" EVALUATING = "evaluating" SCORING = "scoring" DONE = "done" def transition(current_state, event): # The FSM owns "what happens next" — # the LLM only fills in content within a state if current_state == InterviewState.CANDIDATE_CODING: if event == "idle_35s": return InterviewState.HINT_CHECK if event == "code_submitted": return InterviewState.EVALUATING # ...deterministic, testable, no hallucinated jumps Enter fullscreen mode Exit fullscreen mode Compare that to letting the model implicitly track state through conversation history alone — there's no guarantee, no test coverage, and no way to catch a bad transition before it reaches the user. The part that surprised me I expected this to make the product feel more robotic. It did the opposite. Because the structure is now guaranteed, I could actually let the LLM be more creative and natural within each state, without worrying about it going off the rails. Constraining the skeleton let me loosen up the muscle. Counterintuitive, but it checks out — a lot of "unpredictable AI" complaints aren't really about the model being too creative, they're about the model having too much control over things it was never designed to control. The takeaway If you're building something where an LLM is orchestrating a multi-step process — not just answering one-off questions — ask yourself: does the model actually need to decide what happens next, or does it just need to generate good content within a step someone else decides? Most of the time, in my experience, it's the second one. And the moment I stopped asking the LLM to be both the actor and the director, my "why did the interview just end after one question" bugs basically disappeared overnight. I write about the messier, more practical side of building AI products — the stuff that doesn't make it into the demo. If you're curious, I built this exact system as a live product: AI Technical Interviewer. Happy to talk through the architecture more in the comments.