Lesson 5 of 7 15 min

External Memory

How automation systems remember. Persistence patterns for agents.

The Problem: AI Agents Have No Memory

Here's something that surprised me when I started building with AI agents:

Every conversation starts from zero.

Yesterday you told Claude Code about your project structure. Today? Gone. You added three tasks to a todo list. Restart the session? Gone.

AI agents are stateless. They don't remember anything between sessions. All that context you built up? Evaporates the moment you close the terminal.

The Solution: External Memory

External memory is how automation systems remember. Instead of asking the agent to remember, you store state outside the agent:

Session 1:
  You: "Add a task: review PR #42"
  Agent: (calls task_add tool)
  Tool: (saves task to ~/.tasks/tasks.json)
  
  Session ends. Agent forgets everything.

Session 2 (next day):
  You: "What's on my task list?"
  Agent: (calls task_list tool)
  Tool: (reads from ~/.tasks/tasks.json)
  Agent: "You have one task: review PR #42"

The agent didn't remember. The automation layer remembered. That's the key insight.

The Pattern Is Simple

External memory works the same way everywhere:

// 1. Load state from storage
const tasks = loadTasks();

// 2. Modify state
tasks.push(newTask);

// 3. Save state back to storage
saveTasks(tasks);

Load → Modify → Save. That's it.

Everything else is implementation details: where you store it, how you format it, how you handle edge cases.

Start With JSON Files

Here's a controversial opinion: for your first automation layer, use JSON files.

Storage Complexity When It Makes Sense
JSON file Low Learning, prototypes, single-user tools
SQLite Medium Production, queries, multiple tools sharing state
Cloud database High Multi-user, team features, scale

You can always migrate later. Don't over-engineer at the start.

The Code You'll Use

Here's the complete external memory implementation for your Task Tracker:

import * as fs from 'fs';
import * as path from 'path';
import * as os from 'os';

// Where state lives
const TASKS_DIR = path.join(os.homedir(), '.tasks');
const TASKS_FILE = path.join(TASKS_DIR, 'tasks.json');

// Make sure the directory exists
function ensureDir() {
  if (!fs.existsSync(TASKS_DIR)) {
    fs.mkdirSync(TASKS_DIR, { recursive: true });
  }
}

// Load all tasks from disk
export function loadTasks(): Task[] {
  ensureDir();
  if (!fs.existsSync(TASKS_FILE)) return [];  // No file? Empty list.
  return JSON.parse(fs.readFileSync(TASKS_FILE, 'utf-8'));
}

// Save all tasks to disk
export function saveTasks(tasks: Task[]) {
  ensureDir();
  fs.writeFileSync(TASKS_FILE, JSON.stringify(tasks, null, 2));
}

Notice three things:

  1. Location: ~/.tasks/ in your home directory, not in the project folder
  2. Format: JSON with pretty-printing — you can open it in any editor
  3. Safety: Handle missing files gracefully (return empty array, not crash)

Mistakes I've Made (So You Don't Have To)

Mistake 1: Asking the agent to remember things

❌ "Remember that I have a task called 'review PR'"

The agent will say "I'll remember that!" and then forget it 
completely the moment you end the session.

Fix: Store state externally. The agent reads and writes files; it doesn't remember.

Mistake 2: Over-engineering storage

❌ "I need PostgreSQL with migrations and connection pooling"

For a learning project? That's weeks of setup for no benefit.

Fix: Start with JSON files. Migrate when you hit actual limits.

Mistake 3: Crashing on missing files

// ❌ This throws an error if the file doesn't exist
const tasks = JSON.parse(fs.readFileSync(TASKS_FILE, 'utf-8'));

// ✓ This handles the first run gracefully
if (!fs.existsSync(TASKS_FILE)) return [];
return JSON.parse(fs.readFileSync(TASKS_FILE, 'utf-8'));

How Production Systems Do It

Your Task Tracker uses JSON files. Production systems use the same pattern with different storage:

System What It Does Storage Pattern
Loom Task coordination SQLite + periodic checkpoints
Ground Code verification JSON evidence files per run
WORKWAY Workflow automation Cloudflare D1 (SQLite at the edge)

The pattern is identical. Only the backend changes.

Two Kinds of External Memory

You've seen one kind: data memory. Tasks saved to tasks.json. State that persists between sessions.

There's a second kind: spatial memory. And it's just as important.

Spatial Memory: CLAUDE.md

CLAUDE.md is a file that lives at the root of your project. Claude Code reads it automatically at the start of every session.

# My Project

## Architecture
- Frontend: SvelteKit at packages/web/
- API: Express at packages/api/
- Database: PostgreSQL

## Conventions
- Use TypeScript everywhere
- Test files go in __tests__/ next to the source
- Don't modify node_modules (obviously)

This is external memory applied to the agent's understanding, not just task data.

Without CLAUDE.md: Every session starts from zero. You explain your project structure again and again.

With CLAUDE.md: Every session starts informed. The agent already knows your conventions, your file layout, your constraints.

Same Load → Modify → Save pattern. Same principle. Different target:

Type What It Stores File Who Reads It
Data memory Task state tasks.json Your tools
Spatial memory Project context CLAUDE.md The agent itself

The Session Handoff Pattern

Here's how production work survives session breaks:

Session 1 (morning):
  - Work on auth feature
  - Make decisions (use JWT, not sessions)
  - Modify 3 files
  - Session ends (context limit, lunch, whatever)

Between sessions:
  - End-of-session notes saved to a checkpoint file
  - Key decisions documented
  - File modification summary written

Session 2 (afternoon):
  - Agent reads checkpoint
  - Already knows: "JWT chosen, 3 files modified, next step is tests"
  - Continues without re-explaining everything

This isn't magic. It's the same external memory pattern, applied to the development workflow itself.

The insight: External memory isn't just for your tools. It's for your entire development process. Everything the next session needs to know should live on disk, not in someone's head.

The Triad Applied

How does external memory pass the three questions?

Question Answer
DRY One loadTasks() function, called everywhere. No duplication.
Rams JSON file is the simplest storage that works. Nothing extra.
Heidegger Storage serves the workflow (managing tasks). It belongs here.

Try This Now

Think about what would break if your daily tools forgot everything between sessions:

  • Your terminal history? Gone.
  • Your editor's recent files? Gone.
  • Your browser's bookmarks? Gone.

Everything you thought of — that's what external memory prevents. Your automation layer needs the same thing.

Try This in Claude Code

Copy this prompt and paste it into Claude Code:

Create a CLAUDE.md file for this project. Include:
1. A brief description of what the project does
2. The key directories and what they contain
3. Any conventions I should know about (naming, testing, etc.)
4. Things you should NOT modify

This is spatial memory — context that persists between our sessions.

What you're practicing: Creating external memory for Claude Code itself. Every future session starts informed instead of from zero.

What you should see: Claude Code creates a CLAUDE.md with your project's structure and conventions. On your next session, it will automatically read this file.

Bonus: After Claude Code creates the file, close the session and start a new one. Ask "What do you know about this project?" — it should remember.

Lesson Complete

You've learned:

  • ✓ AI agents are stateless — they forget between sessions
  • ✓ External memory: Load → Modify → Save
  • ✓ Data memory (tasks.json) vs. Spatial memory (CLAUDE.md)
  • ✓ Start with JSON files, migrate when needed

The goal you achieved: You created a CLAUDE.md file that gives Claude Code persistent memory of your project.