# Project 05: Personal Memory System

# source/project.md — Detailed Project Specification

> Generated by: skills/lesson-generator
> For the high-level overview see the root PROJECT.md and the teaching content in
> `source/lesson.agent.md`.
> This file contains implementation contracts, not teaching content.

---

## Definition of Done

The project is complete when all of the following are true:

- [ ] Core implementation runs without errors (score → retrieve top-k → inject → answer → write back)
- [ ] All project milestones M1–M6 demonstrated (see `lesson.agent.md` §7)
- [ ] Retrieval ranks memories on **relevance + recency + importance**, not similarity alone
- [ ] Recency is **exponential decay** `decay_rate ** (hours since last access)`, and `retrieve()`
      **touches** `last_accessed` for the memories it returns
- [ ] Retrieval returns a bounded **top-k** (a context budget), not the whole stream
- [ ] The chat loop **writes each turn back** into the memory stream
- [ ] UNDERSTANDING.md completed before first line of code
- [ ] FAILURE_ANALYSIS.md contains ≥3 intentional experiments (e.g. relevance-only, no decay, no write-back)
- [ ] EVALUATION.md contains quantitative results (score breakdowns, rank changes per ablation), not impressions
- [ ] STARCALLOS_REFLECTION.md identifies at least one concrete applicable pattern
- [ ] Guiding tests in `code/tests/` pass (`python -m pytest`)

---

## File Specification

Eight files in `code/`. Embeddings, the chat call, and the memory data structure are **provided**
(plumbing carried from Projects 01–04), so the friction here is the memory-specific work: the
three-signal **scoring** and the **retrieval** that composes it.

### `config.py` — `provided`

Canonical provider-resolution block (do not edit) + a per-project `Config` adding `decay_rate`
(0.995), `top_k` (5), and the three weights `w_relevance` / `w_recency` / `w_importance` (1.0 each),
exposed together as `cfg.weights == (w_relevance, w_recency, w_importance)`.

### `embedding_helpers.py` — `provided`

**Purpose:** Wrap `litellm` embeddings (same as Projects 02–04). `embed_many` / `embed_one`. The model
is chosen by `config.py`; pin ONE model for memories and query.

### `memory_store.py` — `provided`

**Purpose:** The memory data model and an in-memory stream — the "external context" you retrieve from.

**Key API:** the `Memory` dataclass (`id`, `text`, `kind`, `created_at`, `last_accessed`, `importance`,
`embedding`); `MemoryStore.remember(text, embedding, *, kind, importance, now)`, `.add(memory)`,
`.all()`, `.get(id)`. Kind constants `EPISODIC` / `SEMANTIC` / `PROCEDURAL`.

**Does not:** score or rank (that's `scoring.py` / `retriever.py`).

### `scoring.py` — `learner`

**Purpose:** The core — turn a memory + a query + a clock into a single rank-able number. (M1–M3.)

**Key functions (learner-owned core):**

```python
def recency_score(now: float, last_accessed: float, decay_rate: float = 0.995,
                  unit_seconds: float = 3600.0) -> float:
    """Exponential decay: decay_rate ** (hours since last access). 0h -> 1.0; clamp future skew to <= 1.0."""

def importance_score(importance: float, max_scale: float = 10.0) -> float:
    """Normalize a 1–10 salience rating to [0, 1] (clamped)."""

def relevance_score(query_vec: list[float], mem_vec: list[float]) -> float:
    """Cosine similarity (Project 02). Guard zero-norm -> 0.0."""

def retrieval_score(rel: float, rec: float, imp: float,
                    weights: tuple[float, float, float] = (1.0, 1.0, 1.0)) -> float:
    """Weighted sum: w_rel*rel + w_rec*rec + w_imp*imp."""
```

**Provided:** `main()` demo. **Does not:** touch the store or call a provider — pure math.

**Example I/O:**
```text
recency_score(1_000_000.0, 1_000_000.0)            -> 1.0
recency_score(1_000_000.0, 1_000_000.0 - 3600)     -> 0.995          # 1 hour, 0.995 ** 1
importance_score(10)                                -> 1.0
relevance_score([1,0,0], [0,1,0])                   -> 0.0           # orthogonal
retrieval_score(0.8, 0.9, 0.7)                      -> 2.4           # equal weights
```

### `retriever.py` — `learner`

**Purpose:** Page the top-k worth-remembering memories into the prompt budget. (M4.)

**Key function (learner-owned core):**

```python
def retrieve(store: MemoryStore, query_embedding: list[float], now: float, k: int = 5,
             weights: tuple[float, float, float] = (1.0, 1.0, 1.0),
             decay_rate: float = 0.995) -> list[Memory]:
    """Score every memory (relevance + recency + importance), sort desc, take top-k, and set
    last_accessed = now on the returned memories. Takes the query EMBEDDING (a vector), not text,
    so it's pure/offline-testable."""
```

**Provided:** `main()` demo. **Does not:** embed (the caller/orchestrator does) or call the chat model.

**Example I/O:**
```text
# store: m0 "brushed teeth" imp=1 last_accessed=now-1e6 ; m1 "chose Groq" imp=8 last_accessed=now-10
# both embeddings == [1,0,0]
retrieve(store, [1,0,0], now=now, k=1)  ->  [m1]      # relevance ties; recency+importance win
# afterwards: m1.last_accessed == now   (touched)
```

### `chat_with_memory.py` — `provided`

**Purpose:** The end-to-end loop wiring embed → retrieve → inject → answer → write-back. Calls the
learner modules, so it raises `NotImplementedError` until they're done — then answers using memory and
stores each turn. `--seed` preloads a few memories and asks one question for a quick demo.

### `tests/` — `provided`

`test_scoring.py`, `test_retriever.py` (offline guiding tests, fail until the core is implemented),
`test_config_models.py` (drift guard, passes today), `conftest.py` (puts `code/` on `sys.path`).

---

## Input / Output Contracts

| Function | Input | Expected Output | Error Behavior |
|----------|-------|-----------------|----------------|
| `recency_score(now, last, d)` | `float`, `float`, `float` | `float` in `(0,1]`, `= d**hours` | future skew (last>now) → clamp to ≤ 1.0 |
| `importance_score(i)` | `float` 1–10 | `float` in `[0,1]` (`i/10`) | out-of-range → clamp to `[0,1]` |
| `relevance_score(q, m)` | `list[float]`, `list[float]` | cosine in `[-1,1]` (~`[0,1]` for related) | zero-norm vector → `0.0` |
| `retrieval_score(rel, rec, imp, w)` | 3×`float`, weights | weighted sum `float` | — |
| `retrieve(store, q_vec, now, k, w, d)` | store, vector, clock, budget | top-k `list[Memory]`, ranked desc, returned memories touched | empty store → `[]`; `k ≥ len` → all, ranked |

---

## Extended Requirements

Beyond the core implementation — complete these after the core is working.

- [ ] **LLM-rated importance:** rate each new memory's poignancy 1–10 with the model at write time,
      instead of the fixed default (source: `sources/papers/generative-agents.md`).
- [ ] **Kind-aware decay:** make `SEMANTIC`/`PROCEDURAL` memories resist decay (floor or skip the
      recency term) while `EPISODIC` memories decay (source: `sources/papers/memory-systems-taxonomy.md`).
- [ ] **Reflection / promotion:** periodically synthesize repeated important episodic memories into a
      durable semantic fact and write it back (source: `sources/papers/generative-agents.md`).
- [ ] **Persistence:** persist the stream to disk (JSON) or a Chroma collection (Project 03) so memory
      survives across sessions (source: `sources/papers/memgpt.md` — external context).

---

## Known Difficulty Spikes

Listed so the learner expects them, not so they can avoid them.

1. **Touch-on-retrieve.** The single most-missed step: `retrieve()` must set `last_accessed = now` for
   the memories it returns, *after* scoring. Forget it and recency silently degrades to "age since
   creation," so frequently-used memories decay anyway. The guiding test pins this.
2. **Recency is from last access, not creation.** The whole point of the recency signal is the feedback
   loop with touch-on-retrieve. Scoring from `created_at` is a subtle wrong answer that still "runs."
3. **Relevance-only is not a memory system.** It's easy to stop after cosine similarity (you already
   built that in Project 04). The recency + importance signals are the new work; without them you've
   rebuilt RAG over a chat log.
4. **Importance is write-time, not query-time.** Assign it once when the memory is created; don't
   recompute per query (that conflates importance with relevance).
5. **Same embedding model both sides** (carries from Project 02) — mixing models makes relevance garbage
   and the whole ranking fails silently.

---

## Debugging Approach

When things break, check in this order:

1. Environment — is `.env` loaded? Is a provider available (`USE_OLLAMA=1` or a key)? (Only the chat
   loop needs it; scoring/retrieval run offline.)
2. Scoring — `python -m pytest tests/test_scoring.py`: do the four functions pass? (offline)
3. Recency — print `recency_score` for 0h / 1h / 1wk: does it decay from 1.0 downward?
4. Touch — after `retrieve()`, print the returned memories' `last_accessed`: is it `now`?
5. Ranking — print each candidate's `(rel, rec, imp, score)`: is the memory you expected actually
   scoring highest, and *why* (which signal dominated)?
6. Memory vs prompt — print the messages from the orchestrator: are the retrieved memories actually in
   the prompt, and is the new turn being written back?
7. Source — re-read `source/lesson.agent.md` §3 and `sources/papers/generative-agents.md`.

---

## Integration Notes

**Depends on:** Project 02 (embeddings + cosine similarity — relevance scoring is exactly this; the
same-model-both-sides rule), Project 03/04 (retrieving top-k from a store), Project 01 (chat completion
via LiteLLM; system + user message construction for injecting memories).

**Depended on by:** Project 08 (agent — write-back becomes a tool the model calls; MemGPT self-editing
memory), Project 09 (personal learning OS — reflection: episodic → durable semantic knowledge),
Project 07 (evaluation — generalizes LLM-rated importance into an LLM-as-judge scorer). The
score-rank-retrieve-and-touch skeleton is the reusable memory primitive.