Anatomy of a Production LLM State Machine: A Technical Deep-Dive

Most LLM agents fail in production not because the model is wrong, but because the surrounding system has no memory of where it was, no rules for where it can go next, and no trace of how it got there. A well-designed state machine fixes all three. This tutorial deconstructs a real-world autonomous code-review agent — one that reads a pull request, asks clarifying questions when needed, requests revisions, and ultimately approves or rejects — into its constituent state machine components, then shows you exactly how to implement it with LangGraph, PostgreSQL checkpointing, and LangSmith tracing.

---

1. Defining the Agent’s State Graph

The code-review agent moves through four primary states:

Analyze → RequestClarification → Revise → Approve | Reject

Each edge carries a guard condition — a predicate evaluated against the shared context before the transition is allowed to fire:

Transition	Guard
Analyze → RequestClarification	context.ambiguity_score > 0.7
Analyze → Revise	context.issues_found == True and context.ambiguity_score <= 0.7
Analyze → Approve	context.issues_found == False
RequestClarification → Analyze	context.clarification_received == True
Revise → Approve	context.revision_accepted == True
Revise → Reject	context.revision_attempts >= 3

Guards are pure functions — they inspect the AgentState TypedDict and return a boolean. This keeps your transition logic testable without running the LLM.

Key design principle: States should be narrow. Analyze does nothing but analyze. RequestClarification does nothing but compose and dispatch a question. Mixing concerns across states is the most common source of non-deterministic agent behavior.

---

2. Mapping the Observe → Solve → Verify → Error Pattern

Before writing a single line of LangGraph code, map your states to a canonical cognitive pattern:

• Observe → Analyze: The agent receives the diff, constructs a focused prompt that lists only the context needed for analysis (file paths, changed lines, coding standards), and produces a structured finding report.
• Solve → Revise / RequestClarification: The agent acts on findings. Each state carries a different system prompt — Revise focuses on remediation steps, RequestClarification focuses on asking a single, unambiguous question.
• Verify → Approve: The agent re-reads the revised diff against its original findings and checks completeness.
• Error → Reject: A terminal state triggered by guard overflow (too many revision attempts) or a hard failure in any node.

Context-appropriate prompts are not optional — they are the architectural contract. When Revise accidentally receives the Analyze system prompt, the model produces analysis instead of a revision plan. The state machine enforces prompt hygiene by construction.

---

3. LangGraph Implementation Walkthrough

Start with the shared state container:

from typing import TypedDict, Optional

class AgentState(TypedDict):
    pr_diff: str
    findings: Optional[list[dict]]
    clarification_question: Optional[str]
    clarification_response: Optional[str]
    ambiguity_score: float
    issues_found: bool
    revision_attempts: int
    revision_accepted: bool
    final_verdict: Optional[str]  # "approve" | "reject"

Each node is a plain Python function that receives the full state and returns a partial update:

def analyze_node(state: AgentState) -> dict:
    result = llm.invoke(ANALYZE_PROMPT.format(diff=state["pr_diff"]))
    parsed = parse_findings(result)
    return {
        "findings": parsed.issues,
        "ambiguity_score": parsed.ambiguity_score,
        "issues_found": len(parsed.issues) > 0,
    }

def request_clarification_node(state: AgentState) -> dict:
    question = llm.invoke(CLARIFY_PROMPT.format(findings=state["findings"]))
    dispatch_to_author(question)  # Slack, GitHub comment, etc.
    return {"clarification_question": question}

Wire the graph with conditional edges driven directly by your guard predicates:

from langgraph.graph import StateGraph, END

def route_after_analyze(state: AgentState) -> str:
    if state["ambiguity_score"] > 0.7:
        return "request_clarification"
    if state["issues_found"]:
        return "revise"
    return "approve"

def route_after_revise(state: AgentState) -> str:
    if state["revision_accepted"]:
        return "approve"
    if state["revision_attempts"] >= 3:
        return "reject"
    return "revise"  # loop back

graph = StateGraph(AgentState)
graph.add_node("analyze", analyze_node)
graph.add_node("request_clarification", request_clarification_node)
graph.add_node("revise", revise_node)
graph.add_node("approve", approve_node)
graph.add_node("reject", reject_node)

graph.set_entry_point("analyze")
graph.add_conditional_edges("analyze", route_after_analyze)
graph.add_conditional_edges("revise", route_after_revise)
graph.add_edge("request_clarification", "analyze")
graph.add_edge("approve", END)
graph.add_edge("reject", END)

app = graph.compile(checkpointer=...)

The router functions are your guard conditions — readable, unit-testable, and entirely decoupled from the LLM calls inside nodes.

---

4. Adding PostgreSQL Checkpointing

Every node execution must be durable. When a long-running review workflow crashes between RequestClarification and Analyze, you need to resume from exactly the state before the crash — not restart from scratch.

LangGraph ships a PostgresSaver checkpointer that serializes the full AgentState to a Postgres table after every node completes:

from langgraph.checkpoint.postgres import PostgresSaver
import psycopg

conn = psycopg.connect(DATABASE_URL)
checkpointer = PostgresSaver(conn)
checkpointer.setup()  # creates checkpoint tables on first run

app = graph.compile(checkpointer=checkpointer)

# Resume a specific thread:
config = {"configurable": {"thread_id": "pr-4821"}}
result = app.invoke(None, config=config)  # None = resume from last checkpoint

The thread_id maps to a PR number or review session ID. This gives you time-travel debugging for free: you can replay any historical workflow from any prior checkpoint by loading its snapshot and re-invoking specific nodes. This is invaluable when a model upgrade changes behavior — you can bisect exactly which state transition produced the regression.

---

5. Wiring LangSmith Tracing to Node Transitions

Checkpointing tells you where the agent was. Tracing tells you why it made each decision. LangSmith integrates with LangGraph via OpenTelemetry-compatible structured events emitted on every node entry and exit.

Enable it with two environment variables:

export LANGCHAIN_TRACING_V2=true
export LANGCHAIN_API_KEY=ls__...

For deterministic bug reproduction, emit a structured event at each guard evaluation:

from langsmith import traceable

@traceable(name="route_after_analyze", tags=["guard", "transition"])
def route_after_analyze(state: AgentState) -> str:
    decision = _route_logic(state)
    # LangSmith captures inputs (state snapshot) and output (decision string)
    return decision

This produces a trace tree where every state transition is a labeled span with its full input context. When a guard misfires in production, you open the trace, find the span, and see the exact ambiguity_score that triggered the wrong branch — no log archaeology required.

---

Putting It All Together

A production LLM state machine is defined by four properties: named states with focused, context-appropriate prompts; explicit guard conditions that control every transition; durable checkpointing that makes every node execution crash-safe and time-travelable; and structured tracing that makes every guard evaluation reproducible.

LangGraph, PostgreSQL, and LangSmith each handle one layer of this contract. The discipline of keeping them separate — no business logic in the checkpointer, no state mutation in the router — is what separates agents that work in demos from agents that run reliably in production.