# Project 06: AI Coding Copilot

# source/project.md — Detailed Project Specification

> Generated by: skills/lesson-generator
> For the high-level overview see the root PROJECT.md and the teaching content in
> `source/lesson.agent.md`.
> This file contains implementation contracts, not teaching content.

---

## Definition of Done

The project is complete when all of the following are true:

- [ ] Core implementation runs without errors (inject context → loop: call → dispatch tools → feed results back → answer)
- [ ] All project milestones M1–M6 demonstrated (see `lesson.agent.md` §7)
- [ ] Every file-path tool routes through `safe_resolve`; a traversal path (`../../etc/passwd`) **fails closed** with `ValueError`
- [ ] `parse_tool_calls` returns `[]` when the model answers, and otherwise `json.loads` each call's `arguments` **string** into a dict
- [ ] `run_agent` appends the **assistant turn before** the tool-result messages, and is **bounded by `max_steps`** (no infinite loop)
- [ ] The copilot answers using files it actually **read via tools** (ideally with a file/line citation), not a guess
- [ ] UNDERSTANDING.md completed before first line of code
- [ ] FAILURE_ANALYSIS.md contains ≥3 intentional experiments (e.g. remove the cap, remove the sandbox, stop feeding results back)
- [ ] EVALUATION.md contains concrete results (tools called, whether the answer cited real files, hallucination check), not impressions
- [ ] STARCALLOS_REFLECTION.md identifies at least one concrete applicable pattern
- [ ] Guiding tests in `code/tests/` pass (`python -m pytest`)

---

## File Specification

Seven files in `code/` (plus tests). The chat call, the embeddings/retrieval (context injection), the
file-op tools, and the orchestrator are **provided** (plumbing carried from Projects 01–04), so the
friction here is the genuinely new work: the **tool-use loop** and its **sandbox**.

### `config.py` — `provided`

Canonical provider-resolution block (do not edit) + a per-project `Config` adding `embedding_model`,
`repo_root` (`"."`), `top_k_files` (4), and `max_agent_steps` (8). Temperature defaults low (0.2) for
deterministic code answers.

### `context_selector.py` — `provided`

**Purpose:** Context injection — Project 02 (cosine) + Project 03/04 (retrieval) applied to source
files. `select_context(repo_root, query, k)` embeds the query and each file with the **one**
`config.embedding_model`, ranks by cosine, and returns the top-k `(relative_path, file_text)` pairs.
Not the learning target; provided so effort goes to the loop.

### `tools.py` — `partial`

**Purpose:** The copilot's action space + its security boundary. (M1, M3.)

**Provided:** `TOOL_SCHEMAS` (the `name`/`description`/`parameters` definitions the model programs
against — the ACI); `read_file` / `list_directory` / `search_code` (sandboxed file ops; the path ops
call `safe_resolve`).

**Key functions (learner-owned core):**

```python
def safe_resolve(repo_root, rel_path) -> Path:
    """Resolve rel_path under repo_root; raise ValueError if it escapes. Resolve THEN check
    containment (base == target or base in target.parents). Untrusted model input — fail closed."""

def dispatch_tool(name: str, arguments: dict, repo_root) -> str:
    """Route a parsed tool call to read_file/list_directory/search_code; return its string result.
    Unknown tool -> an error string (don't raise — a bad name shouldn't kill the loop)."""
```

**Example I/O:**
```text
safe_resolve("/work/repo", "src/config.py")     -> Path("/work/repo/src/config.py")
safe_resolve("/work/repo", "../../etc/passwd")  -> ValueError
dispatch_tool("read_file", {"path": "config.py"}, repo)  -> "<contents of config.py>"
dispatch_tool("frobnicate", {}, repo)                    -> "Error: unknown tool 'frobnicate'"
```

### `agent_loop.py` — `learner`

**Purpose:** The core — read the model's tool requests and drive the ReAct loop. (M2, M4.)

**Provided:** the `ToolCall` dataclass (`id`, `name`, `arguments`).

**Key functions (learner-owned core):**

```python
def parse_tool_calls(message) -> list[ToolCall]:
    """Extract (id, name, arguments) from an OpenAI-style message. arguments is a JSON STRING —
    json.loads it into a dict. No tool_calls -> [] (the model answered)."""

def run_agent(messages: list, repo_root, complete, *, tools=TOOL_SCHEMAS, max_steps: int = 8) -> str:
    """Loop up to max_steps: msg = complete(messages, tools); calls = parse_tool_calls(msg);
    if no calls -> return msg.content; else append msg, then dispatch each call and append a
    {'role':'tool','tool_call_id':id,'content':result} message; repeat. Cap -> a sentinel string.
    `complete` is injected so the loop is testable offline."""
```

**Example I/O:**
```text
# complete returns a read_file call, then a final answer:
run_agent(msgs, repo, scripted_complete, max_steps=5)  -> "...the final answer..."
# a complete that always asks for a tool:
run_agent(msgs, repo, always_calls, max_steps=3)       -> "[stopped: hit max_steps=3 ...]"
parse_tool_calls(msg_with_one_call)   -> [ToolCall("c1", "read_file", {"path": "a.py"})]
parse_tool_calls(msg_with_content)    -> []
```

### `copilot.py` — `provided`

**Purpose:** The orchestrator wiring context injection + the loop. Builds the system prompt, injects
the top-k relevant files, then runs `run_agent` with a REAL LiteLLM completion (`tool_choice="auto"`)
so the model can read more via tools. Calls the learner modules, so it raises `NotImplementedError`
until they're done — then answers questions about a repo end to end.
`python copilot.py "question" --repo .`

### `tests/` — `provided`

`test_tools.py` (M1 safe_resolve, M3 dispatch), `test_agent_loop.py` (M2 parse, M4 loop — a fake
`complete` scripts the model), `test_config_models.py` (drift guard, passes today), `conftest.py`
(puts `code/` on `sys.path` and builds a throwaway fixture repo). All offline — no provider, no
network.

---

## Input / Output Contracts

| Function | Input | Expected Output | Error Behavior |
|----------|-------|-----------------|----------------|
| `safe_resolve(root, rel)` | `str/Path`, `str` | `Path` under `root` (resolved) | escapes root → `ValueError` |
| `dispatch_tool(name, args, root)` | `str`, `dict`, root | tool result `str` | unknown name → `"Error: unknown tool …"`; bad path → file op returns `"Error: …"` |
| `parse_tool_calls(message)` | OpenAI-style message | `list[ToolCall]` (args `json.loads`'d to dict) | no `tool_calls` → `[]` |
| `run_agent(messages, root, complete, …)` | messages, root, completion fn | final answer `str` | hits `max_steps` → sentinel `str`; never raises on a tool error |
| `select_context(root, query, k)` | root, `str`, `int` | up to k `(path, text)` pairs, ranked | no files / no provider → `[]` (copilot falls back to tools) |

---

## Extended Requirements

Beyond the core implementation — complete these after the core is working.

- [ ] **Reasoning trace (full ReAct):** prompt the model to emit a short *Thought* before each action and surface the Thought→Action→Observation trace (source: `sources/papers/react-paper.md`).
- [ ] **A write tool (with care):** add `write_file` / apply-a-diff — sandboxed via `safe_resolve`, with a dry-run/confirmation. Now the copilot edits, not just reads.
- [ ] **Structured output via a tool:** add an `emit_answer(answer, citations[])` tool and force `tool_choice` to it so the final answer is schema-validated with file:line citations (source: `sources/official-docs/anthropic-tool-use.md`).
- [ ] **Parallel tool calls:** handle a turn that requests several tools at once; execute and feed back all results before looping.
- [ ] **Better code retrieval:** chunk files and embed per-chunk instead of per-file (source: `sources/articles/chunking-strategies.md`).

---

## Known Difficulty Spikes

Listed so the learner expects them, not so they can avoid them.

1. **`arguments` is a JSON string.** The model's call carries `function.arguments` as a *string*; you must `json.loads` it before dispatch. Skipping this hands the tool a string and is the #1 first bug.
2. **Message ordering.** The assistant message that *requested* the tools must be appended **before** the `tool` result messages, or the provider rejects the next call (orphan tool result). The happy-path demo won't catch this until you run it live.
3. **The loop must be bounded.** The model drives control flow; a confused model requests tools forever. `while True` is a runaway-budget bug — cap at `max_steps` and return cleanly.
4. **Sandbox after resolving.** `safe_resolve` must resolve the path *then* check containment. A raw-string `".." in path` check is unsound (symlinks, absolute paths slip through).
5. **Don't crash the loop on a tool error.** A bad path or unknown tool should return an error *string* the model can read and react to (ReAct observation), not raise and kill the run.
6. **Answering without reading is failure, not success.** A copilot that replies instantly with no tool call has guessed. Verify it actually read the file (this is the ReAct hallucination point).

---

## Debugging Approach

When things break, check in this order:

1. Environment — is `.env` loaded? (Only `copilot.py`'s live run needs a provider; all tests run offline.)
2. Tools — `python -m pytest tests/test_tools.py`: do `safe_resolve` (M1) and `dispatch_tool` (M3) pass?
3. Sandbox — call `safe_resolve(repo, "../x")`: does it raise? Call it on a valid file: does it return a real `Path`?
4. Parsing — `python -m pytest tests/test_agent_loop.py -k parse`: is `arguments` a **dict** (json.loads'd), and `[]` when the model answers?
5. Loop — does `run_agent` return the final answer when `complete` stops calling tools, and the **sentinel** when a model always calls? Did you append the assistant message before the tool results?
6. Live — `python copilot.py "list the python files" --repo .`: does it call `list_directory`, then answer? Print `messages` to see the tool turns interleave correctly.
7. Source — re-read `source/lesson.agent.md` §3 and `sources/official-docs/anthropic-tool-use.md`.

---

## Integration Notes

**Depends on:** Project 01 (chat completion via LiteLLM; the loop's inner call is `litellm.completion`
+ `tools=`), Project 02 (cosine similarity — the context-injection ranking), Project 03/04 (retrieving
top-k from a store — context injection applied to source files; the citation/faithfulness discipline).

**Depended on by:** Project 07 (evaluation — measuring copilot correctness, relevance, and
hallucination over tool use), Project 08 (agent — this loop generalized: more tools, planning,
sub-tasks, self-editing memory). The tool-use protocol (schema → call → dispatch → result → loop)
built here is the substrate for every later agent; MemGPT's "self-editing memory via tools" (Project
05's forward link) is this mechanism pointed at the memory store.