AI Coding Copilot

Learning Objectives

By the end of this project, you will be able to:

• Explain why an LLM with no tools can only guess about your codebase, and why tools are "one of the highest-leverage primitives you can give an agent."
• Define a tool as a name + description + JSON-schema parameters, and explain why "the description is the API."
• Trace the tool-use protocol: the model returns a structured call, your code executes it, you feed the result back, the model continues.
• Implement the agentic loop — call → dispatch tools → feed results back → repeat — and explain why the loop, not a clever prompt, is the program.
• Explain ReAct (interleaved reasoning + acting) and why acting reduces the hallucination a reasoning-only prompt produces.
• Distinguish context injection (retrieve relevant files up front) from tool-fetched context (the model pulls what it needs), and say when each wins.
• Parse provider tool calls correctly — including that arguments arrives as a JSON string — and bound the loop with max_steps.
• Name the major failure modes of a tool-using copilot (runaway loop, path traversal, vague descriptions, no result feedback) and which decision causes each.

1. Motivation

Why This Exists

An LLM is a pure text function: it cannot read your files, list your directory, or run your tests. Ask a bare model "why does auth.py throw on login?" and it has never seen auth.py — so it guesses, fluently and wrongly. The two non-answers before tool use were: paste your whole repo into the prompt (blows the context window and the bill), or accept generic advice untethered from your code. Tool use is the fix: give the model a few functions — read a file, list a directory, search the code — and let it pull exactly the context it needs.

What Breaks Without It

A tool that takes a file path is an attack surface — an unsandboxed read_file will happily return ../../../../etc/passwd. A loop with no cap spins forever, burning tokens. A vague tool description makes the model call the wrong tool. None of these show up in the happy-path demo — they show up in production.

Real-World Stakes

This is the most widely adopted AI product category: GitHub Copilot, Cursor, Claude Code, and every internal "ask our codebase" tool are this pattern. On real software-engineering benchmarks like SWE-bench, "adding even basic tools produces outsized capability gains."

2. Mental Model

Explain Like I'm 12

Imagine you text a very smart friend "why won't my game start?" — but they can't see your computer. They'll guess, and probably guess wrong. Now imagine you give them three buttons they can press that you answer for them: "show me a file," "list a folder," and "search for a word." Now the conversation changes: they press "list folder," you tell them what's there; they press "show me game.py," you paste it; they read it and say "line 12 — you forgot to load the level." They didn't get smarter — they got eyes. The whole project is: (1) build those buttons, (2) let your friend keep pressing buttons until they actually know the answer, and (3) make sure "show me a file" can't be tricked into showing them your password file.

Explain Like I'm a Software Engineer

• A tool is a function you expose to the model as a schema: {"type":"function","function":{"name","description","parameters"}}. The model never sees the body — only the schema.
• You call litellm.completion(messages, tools=...). The model replies one of two ways: plain content (it's done), or message.tool_calls — a list of {id, function:{name, arguments}} where arguments is a JSON string.
• The loop: parse the calls → execute each (dispatch_tool(name, json.loads(arguments), repo_root)) → append one {"role":"tool","tool_call_id":id,"content":result} per call → call the model again. Repeat until it returns content with no tool calls, or you hit max_steps.
• This is ReAct: the model's text is the Thought, the tool call is the Action, the tool result is the Observation you feed back. Every file-taking tool routes through safe_resolve(repo_root, path), which fails closed on escapes.

Real-World Analogy

How It Works (Diagram)

 USER: "Why does config default to Groq?"
   │  (optionally: inject top-k relevant files as starting context — P03/P04 retrieval)
   ▼
 run_agent loop  (max_steps cap):
   step 1: model ──► tool_call: search_code(query="default_model")
           dispatch ─► safe_resolve+grep ─► "config.py:40 def default_model(): ..."
           append tool_result ──► loop
   step 2: model ──► tool_call: read_file(path="config.py")
           dispatch ─► safe_resolve+open ─► "<contents of config.py>"
           append tool_result ──► loop
   step 3: model ──► (no tool calls) "It defaults to Groq because PROVIDER_DEFAULTS
                      lists GROQ_API_KEY first; see config.py:23." ◄── ANSWER

3. Technical Explanation

Formal Definition

A tool is (name, description, parameters) where parameters is a JSON Schema; the set of tools is the model's action space. A tool call is (id, name, arguments) emitted by the model — in the LiteLLM/OpenAI shape, an entry in message.tool_calls with function.arguments as a JSON string. A tool result is the value your code returns for a given tool_call_id, appended as a {"role":"tool", ...} message. An agent is "an LLM using tools based on environmental feedback in a loop."

How It Works Step by Step

1. Define the tools. Write TOOL_SCHEMAS: a name, a careful description (the model picks from this alone), and a parameters JSON Schema. Implement the matching functions.
2. (Optional) Inject context. Embed the question, retrieve the top-k most relevant files (Project 03/04), and prepend them — cheap grounding so the model doesn't have to discover obvious files via tools.
3. Call the model. litellm.completion(messages, tools=TOOL_SCHEMAS, tool_choice="auto"). With auto, the model decides each turn whether to call a tool or answer.
4. Parse tool calls. If message.tool_calls is non-empty, build (id, name, arguments) and json.loads the arguments string. If empty, the model answered — return its content.
5. Execute (dispatch). For each call, dispatch_tool(name, arguments, repo_root) routes to the right function. Every path argument goes through safe_resolve first.
6. Feed results back, then loop. Append the assistant message (with its tool_calls), then one tool message per call — order matters. Go back to step 3, bounded by max_steps.

Key Concepts

4. Guided Examples

The lab stack: litellm (chat + tools via config.py), provided sandboxed file tools, and a pure-Python loop. Examples mirror the guiding tests.

Example 1: Simplest Case — sandbox a tool path

from tools import safe_resolve

repo = "/work/myrepo"
safe_resolve(repo, "src/config.py")     # -> Path("/work/myrepo/src/config.py")  (ok)
safe_resolve(repo, "./README.md")       # -> Path("/work/myrepo/README.md")       (ok)
safe_resolve(repo, "../../etc/passwd")  # -> raises ValueError  (escapes repo root)

Example 2: Real-World Case — parse the model's tool calls

from types import SimpleNamespace
from agent_loop import parse_tool_calls

# Shape LiteLLM returns: message.tool_calls[i].function.arguments is a JSON *string*.
msg = SimpleNamespace(content=None, tool_calls=[
    SimpleNamespace(id="call_1", function=SimpleNamespace(
        name="read_file", arguments='{"path": "config.py"}')),
])
calls = parse_tool_calls(msg)
print(calls[0].name)        # 'read_file'
print(calls[0].arguments)   # {'path': 'config.py'}   ← a dict, json.loads'd from the string
print(parse_tool_calls(SimpleNamespace(content="done", tool_calls=None)))  # []  (model answered)

Example 3: When It Fails — the loop without a cap runs forever

from agent_loop import run_agent
from types import SimpleNamespace

# A broken/looping model that ALWAYS asks to read the same file, never answers:
def always_calls_tool(messages, tools):
    return SimpleNamespace(content=None, tool_calls=[
        SimpleNamespace(id="c", function=SimpleNamespace(
            name="read_file", arguments='{"path": "config.py"}'))])

answer = run_agent(messages=[{"role":"user","content":"hi"}],
                   repo_root=".", complete=always_calls_tool, max_steps=3)
print(answer)   # a "max steps reached" sentinel — NOT an infinite loop

5. Reflection Before Building

After filling in UNDERSTANDING.md, use the AI mentor pattern in docs/meta/learning-flow.md to get feedback. Record the feedback in UNDERSTANDING_FEEDBACK.md.

6. Project Assignment

See PROJECT.md and source/project.md for the full specification including file structure and extended requirements.

Core Requirement

Build a context-aware coding copilot in code/ that answers questions about a target repository by reading it through tools:

• tools.py — safe_resolve (path sandbox) and dispatch_tool (route a tool call) are the learner core; read_file / list_directory / search_code and TOOL_SCHEMAS are provided.
• agent_loop.py — parse_tool_calls (extract calls, json.loads the args) and run_agent (the ReAct loop) are the learner core — the learning target.
• context_selector.py — embed the question, rank files by cosine, return top-k as starting context. (Provided — Project 02/03 reuse.)
• copilot.py — the orchestrator: build the prompt, inject context, run the agent, print the answer. (Provided.)

Extended: a reasoning trace (full ReAct); a sandboxed write_file with dry-run; structured output via a forced emit_answer tool with citations; parallel tool calls.

7. Project Milestones

Work through these in order. Each milestone should produce runnable code before moving on.

8. Self-Evaluation

After building, honestly evaluate your implementation against these criteria. Record your answers in EVALUATION.md.

9. Common Mistakes

10. Connections

Builds On

The loop's inner call is Project 01's litellm.completion — you've just added tools= and wrapped it in a while. The context injection half (embed the question, rank files by cosine, take top-k) is Project 02's similarity and Project 03/04's retrieval applied to source files, unchanged — which is why this lesson leaves it provided and spends your effort on the genuinely new thing: the tool loop.

Enables

Project 07 (evaluation) is how you'd measure this copilot — correctness, relevance, and whether it hallucinated vs. read the file. Project 08 (agent) is this loop generalized: more tools, planning, sub-tasks, and self-editing memory — "LLMs using tools in a loop" scaled up. The tool-use protocol you build here is the substrate for every later agent.

Production Patterns

Real copilots are this loop plus engineering: retrieval to seed context, a curated tool set with carefully documented schemas (the ACI), strict sandboxing and permissioning on every tool, a bounded agentic loop with retries and a step budget, structured/cited final output, and evaluation in the loop. GitHub Copilot, Cursor, and Claude Code are all variations on "augmented LLM (retrieval + tools) run in a loop" with production-grade interfaces and guardrails.

StarcallOS Relevance

Sources

See source/resources.md for the complete annotated source list.