AI Agent

Learning Objectives

By the end of this project, you will be able to:

• State the workflow vs. agent distinction — predefined code paths vs. the LLM directing its own process — and decide which a task needs.
• Explain why an autonomous, multi-step loop is fundamentally riskier than P06's single-question copilot: the model now controls how many steps, which tools, and when to stop.
• Implement loop/stuck detection, and explain why a max_steps cap alone misses an agent that oscillates with no progress.
• Implement a budget over steps and tokens, and explain why an autonomous agent without one is a runaway-cost incident waiting to happen.
• Implement tool-error recovery — catch a failing tool, feed the error back as an observation, let the model adapt — and why crashing wastes every prior step.
• Evaluate an agent run quantitatively — completion, steps, tool calls, efficiency — instead of eyeballing "it seemed to work."
• Apply "add complexity only when it demonstrably improves outcomes" — name tasks where a single call or a fixed workflow beats an agent.
• Name the major failure modes of an autonomous agent (runaway cost, productive-looking oscillation, one tool error killing the run, agent-as-overkill) and which decision prevents each.

1. Motivation

Why This Exists

Project 06's copilot answers one question: it reads a few files and replies. But "refactor this module and make the tests pass," "research this topic and write a summary," or "triage this bug" are not one question — they are open-ended tasks where you "can't predict the required number of steps." That is exactly what an agent is for: the model plans, acts, observes, and decides for itself when it's done. But the moment you hand the model control over the loop, you hand it control over your bill, your correctness, and whether the program ever terminates.

What Breaks Without It

The single hardest thing in AI engineering is not making an agent that works on the happy path — it's making one that fails safely on every other path. "Users forgive bad suggestions, but not autonomous actions that cause damage." A copilot that suggests a wrong line costs you nothing; an agent that loops 200 times overnight costs real money, and one that acts without a guard is an incident.

Real-World Stakes

This is the product category everyone is racing to ship: autonomous coding agents, research agents, computer-use agents, customer-ops agents. Every production agent has the machinery you build here — step/token budgets, loop detection, retries with recovery, and an evaluation harness — because without it the agent is a liability, not a feature. "The most successful implementations use simple, composable patterns."

2. Mental Model

Explain Like I'm 12

In Project 06 you gave a friend who can't see your computer some buttons to press ("show me a file," "search"), and they pressed buttons until they answered one question. Great for one question. Now give that friend a whole chore — "clean up my messy folder" — and leave the room. Three bad things can happen. One: they get stuck, opening the same drawer over and over, forever. Two: they never stop — they work for ten hours and you get a giant bill. Three: one drawer is jammed, and instead of skipping it they give up on the whole chore. A good helper notices "I keep opening this same drawer — let me stop," has a rule like "you get 20 minutes," and when a drawer jams says "okay, skip it, I'll do the rest." This project teaches your friend those three habits — and teaches you the wisdom to not send a friend to do a chore you could've done yourself in one move.

Explain Like I'm a Software Engineer

• An agent is P06's loop — call model → dispatch tools → feed results back → repeat — but for open-ended tasks. The loop, tools, parsing, and sandbox are provided (you built them in P06). The learning target is the control layer.
• Stuck detection (detect_stuck(history)): keep a history of (tool, arguments) actions; if the last N are identical, stop now — don't wait for max_steps.
• Budget (BudgetTracker): a hard ceiling on steps and tokens, tick()'d each step and checked with over_budget(). It's what makes "autonomous" not mean "unbounded."
• Recovery: dispatch each tool inside try/except; on failure, feed f"Error: {e}" back as the observation. "Reasoning traces help the model handle exceptions" — it reads the error and adapts.
• Evaluation (evaluate_run): a run produces a RunResult (answer, history, steps, tokens, stop_reason). Score it: did it complete? how many steps vs. optimal? at what cost?
• Judgment: before any of this, ask whether the task needs an agent at all. "Find the user's email" is a function call. "Summarize this PDF" is one retrieval-augmented call (P04). The loop is for tasks whose step count you genuinely can't predict.

Real-World Analogy

How It Works (Diagram)

 TASK: "What model does this repo default to, and where is it set?"  (multi-step)
   │
   ▼  run_agent(budget = steps≤10, tokens≤20k)
   step 1: model ─► search_code("default_model")     ── budget.tick ── history=[search]
   step 2: model ─► read_file("config.py")            ── budget.tick ── history=[search, read]
           (tool raises? → caught → "Error: ..." fed back → model adapts)   ← RECOVERY
   step 3: model ─► (no tool call) "Defaults to groq/llama-3.3-70b; config.py:24." ─► RETURN answered

 vs. a BROKEN run on a confusing task:
   step 1..3: model ─► search_code("x") x3            ← detect_stuck → STOP (stuck)
   ... or 50 productive-looking steps with no end     ─► budget.over_budget() → STOP (budget)

 evaluate_run(result, expected={answer_contains:"groq", optimal_steps:3})
   ─► {completed: True, steps: 3, tool_calls: 2, efficiency: 1.0, stop_reason: "answered"}

3. Technical Explanation

Formal Definition

A workflow orchestrates LLMs and tools "through predefined code paths." An agent is a system where "LLMs dynamically direct their own processes and tool usage." An agent run is a sequence of steps; each step is one model call that yields a final answer or one-or-more tool calls. The run has a stop reason ∈ {answered, stuck, budget, max_steps}. A run is stuck when the last N actions are identical (tool, arguments). A run is over budget when cumulative steps or tokens cross a pre-set ceiling. Recovery converts a raising tool into an error observation fed back into the conversation.

How It Works Step by Step

1. Decide if you even need a loop. Known, fixed shape (one lookup, one summary, classify-then-route)? Use a single call or a workflow. Reach for the agent only when "it's difficult or impossible to predict the required number of steps."
2. Set a budget. Construct a BudgetTracker(max_steps, max_tokens) before the run — the contract for what the agent may spend.
3. Run the loop. Each iteration: call the model, tick the budget with the step's tokens, parse_tool_calls (provided). No calls → answered. Otherwise append the assistant turn, then for each call: record the action, dispatch_tool it inside try/except (recovery), append the result/error as a tool message.
4. Guard after each step. detect_stuck(history) → stop (stuck). budget.over_budget() → stop (budget). Loop exhausts its structural bound → max_steps.
5. Evaluate the run. Pass the RunResult + an expected spec to evaluate_run: completion, steps, tool calls, efficiency. Now a prompt or model change can be scored, not guessed at.

Key Concepts

4. Guided Examples

The lab stack: litellm (chat + tools via config.py), the provided P06 tool layer (safe_resolve, parse_tool_calls, dispatch_tool, file tools), and a pure-Python loop. Examples mirror the guiding tests.

Example 1: Simplest Case — detect a stuck agent

from safety import Action, detect_stuck

# An agent making progress: three DIFFERENT actions → not stuck.
progress = [Action("search_code", {"query": "x"}),
            Action("read_file", {"path": "a.py"}),
            Action("read_file", {"path": "b.py"})]
detect_stuck(progress, window=3)   # -> False

# An agent spinning: the last three actions are IDENTICAL → stuck.
spinning = [Action("read_file", {"path": "a.py"}),
            Action("search_code", {"query": "foo"}),
            Action("search_code", {"query": "foo"}),
            Action("search_code", {"query": "foo"})]
detect_stuck(spinning, window=3)   # -> True

Example 2: Real-World Case — enforce a budget

from safety import BudgetTracker

b = BudgetTracker(max_steps=5, max_tokens=10_000)
b.tick(2_000)          # step 1
b.tick(2_000)          # step 2
b.over_budget()        # -> None        (2 steps, 4k tokens — within budget)

b.tick(7_000)          # step 3, now 11k tokens
b.over_budget()        # -> "token budget exhausted (11000/10000)"   (a reason string — truthy)

Example 3: When It Fails — one tool error must not kill the run

from types import SimpleNamespace
from safety import BudgetTracker
from agent import run_agent
import agent

# Force the (provided) dispatch to RAISE on the first call, as a flaky real tool would:
calls = {"n": 0}
def exploding_dispatch(name, args, repo_root):
    calls["n"] += 1
    if calls["n"] == 1:
        raise RuntimeError("network down")
    return "ok"
agent.dispatch_tool = exploding_dispatch   # monkeypatch for the demo

turns = iter([
    SimpleNamespace(content=None, tool_calls=[_tc("c1", "read_file", '{"path": "a.py"}')]),  # raises
    SimpleNamespace(content="Recovered and answered.", tool_calls=None),                      # adapts
])
result = run_agent([{"role": "user", "content": "go"}], ".", lambda m, t: next(turns),
                   BudgetTracker(max_steps=5, max_tokens=10_000))
print(result.stop_reason)   # 'answered'  — the raise became an observation, the model adapted

5. Reflection Before Building

After filling in UNDERSTANDING.md, use the AI mentor pattern in docs/meta/learning-flow.md to get feedback. Record the feedback in UNDERSTANDING_FEEDBACK.md.

6. Project Assignment

See PROJECT.md and source/project.md for the full specification including file structure and extended requirements.

Core Requirement

Build the reliability layer that turns P06's tool loop into a trustworthy autonomous agent, in code/:

• safety.py — detect_stuck (M1) and BudgetTracker (M2) are the learner core: the no-progress detector and the step/token ceiling.
• agent.py — run_agent (M3) is the learner core — the generalized loop with tool-error recovery and the stuck/budget guards. RunResult, parse_tool_calls, and the token estimator are provided.
• evaluate.py — evaluate_run (M4) is the learner core: score a run on completion, steps, tool calls, and efficiency.
• tools.py — the entire P06 tool layer. (Provided — you built this in Project 06.)
• agent_app.py — the orchestrator: a real LiteLLM complete, a budget, run the agent on a multi-step task, print the trace + answer + eval. (Provided.)

Extended: explicit planning (decompose before acting); a reflection loop (evaluator-optimizer); structured final output via a forced emit_answer tool; a real max_usd cost budget; retry-with-backoff on transient tool errors.

7. Project Milestones

Work through these in order. Each milestone should produce runnable code before moving on.

8. Self-Evaluation

After building, honestly evaluate your implementation against these criteria. Record your answers in EVALUATION.md.

9. Common Mistakes

10. Connections

Builds On

This is Project 06's loop, made reliable. The tool layer (sandbox, schemas, dispatch), the parsing, and the basic call→dispatch→feed-back cycle are all carried over provided — which is the whole point: you already learned them, so here you spend your effort on the control layer. The budget is Project 01's token-cost arithmetic, enforced live. The evaluation is Project 07's discipline (score, don't eyeball) applied to a run. The optional reflection loop is Project 05's reflection. The agent is where the whole course converges.

Enables

Project 09 (Personal Learning OS) orchestrates memory (P05), retrieval (P03/P04), this agent (P08), and evaluation (P07) into one system. The reliability layer you build here is what makes it safe for P09 to let an agent act over a user's personal data: a bounded, stuck-aware, recoverable agent with an auditable run log. Everything P09 routes to an agent inherits these guards.

Production Patterns

Real agents are this loop plus production engineering: hard budgets on steps/tokens/cost/wall-clock; loop and stuck detection; retries with backoff and graceful recovery; structured, validated final output; a full run trace for observability; and an evaluation harness gating every change. Anthropic's own guidance is to bias toward the simplest pattern that works — single call, then workflow, then agent — and to "invest in the agent-computer interface." The named patterns (prompt chaining, routing, orchestrator-workers, evaluator-optimizer) are the composable pieces; the autonomous loop you build here is the most general and the one to reach for last.

StarcallOS Relevance

Sources

See source/resources.md for the complete annotated source list.