Resources

Token usage

What memory costs per turn in tokens, and how it stays low.

llms.txt

The connector adds a small token overhead to each conversation turn in exchange for persistent memory across sessions.

Per-turn cost of process_turn

Request

~200

tokens (params)

Response

~400

tokens (context + metadata)

Total

~600

tokens per turn

Typical conversation overhead

A 20-turn conversation adds roughly ~12K tokens from MCP calls. Compared to a typical conversation using 50 to 200K tokens total, this represents about 5 to 10% overhead, the cost of giving your agent persistent memory across sessions.

How we keep it low

  • Compact serialization, pipe-delimited format uses 2.3x fewer tokens than raw JSON for the same memories
  • Delta serving, only sends memories that changed since the last turn (~80% savings over 20 turns)
  • Context entries are truncated with memory IDs, the agent can search_memories(id) for full content when needed
  • Recall returns only a bounded set of the most relevant memories (top 10 by default), not everything
  • Embeddings are computed server-side (cloud mode) or locally via Candle (local mode)
  • Other tools (store_memory, search_memories) are called on-demand, not every turn