Facts, procedures, current work and past experience need different rules.

TLDR: Lumping every input into a generic "agent memory" creates failures like stale authority, task leakage, and premature generalization. Production systems require distinct lifecycle and update rules for four separate layers: knowledge (domain facts), skills (versioned procedures), working context (job-scoped state), and memory (unprocessed historical evidence).

An agent is preparing a video for a new campaign. It can access the client’s product rules, a procedure for turning an idea into scenes, decisions made for this campaign and feedback from earlier reviews.

All four may end up in the model’s context. That does not make them the same kind of information.

The distinction matters when the system changes. A campaign instruction should expire with the campaign. A stable product rule should survive across campaigns. A production procedure should change through a versioned and tested release. A reviewer comment should be retained without silently becoming policy.

Many agent architectures already separate similar concepts, although they use different names. CoALA describes semantic, procedural, working and episodic memory. LangGraph uses a related split between facts, procedures, experiences and thread-scoped state.[1][2]

Anthropic discusses context and Agent Skills as separate engineering objects.[3][4]

The names are not standard. The lifecycle differences are useful.

At Replai, we use four categories: knowledge, skills, working context and memory.

Knowledge answers: what is true here?

Knowledge describes the product, client or domain. It can include terminology, product properties, brand rules, market restrictions and approved examples.

Knowledge is stable enough to reuse, but it is not permanent truth. A client can change a requirement. A rule may apply only in one country. Two documents may conflict.

Each knowledge item therefore needs more than text. It needs a source, scope, owner and validity period. When a rule changes, the system should supersede the old version rather than retrieve both as equally valid instructions.

The common failure is stale authority: an old but semantically similar document outranks the current rule.

Skills answer: how do we do this?

A skill is a reusable procedure. It describes how to turn defined inputs into an output using steps, tools, constraints and checks.

Knowledge may say that a product must appear early in a video. A scripting skill describes how to apply that rule while constructing scenes, checking continuity and producing an artifact for review.

Anthropic’s Agent Skills package instructions, scripts and resources so an agent can load them when needed. Anthropic also recommends starting from observed failures and evaluating skills on representative tasks.[4]

That evaluation requirement matters. Changing a skill changes behaviour. Treat the change like code: version it, test it against known cases and keep a rollback path. A reviewer comment is not enough evidence to rewrite a shared procedure.

The common failure is behavioural drift: instructions accumulate until nobody can tell which change caused the regression.

Working context answers: what is happening now?

Working context contains the state of the current job. For a campaign, that might include the brief, chosen concept, target format, intermediate scenes, open questions and decisions already made.

Its defining property is scope. The information must survive the stages of this job without leaking into another one.

Anthropic defines context as the information available to the model during inference. That includes instructions, tools, retrieved data and message history. Because model attention is finite, Anthropic recommends selecting a small set of high-signal information instead of loading everything available.[3]

Stored information is not automatically context. The system constructs context for each decision by selecting relevant knowledge, skills and current state.

The common failure is task leakage: an exception made for one campaign appears as an instruction in the next.

Memory answers: what happened before?

Memory retains experiences that may matter later: outcomes, corrections, failed attempts and user feedback.

This is narrower than how some frameworks use the word. CoALA and LangGraph call facts and procedures forms of memory too.[1][2] We reserve the term for retained evidence from experience because evidence and policy should not share an update path.

Suppose a reviewer says, "The product should appear in the opening scene." The comment matters immediately, but its long-term meaning is unclear. It could be:

• a requirement for this campaign;
• a stable client rule;
• a rule the current skill failed to apply;
• a personal preference that should not generalise.

Memory preserves the observation while the team determines its scope. Learning happens only after the system routes the evidence to the right layer, makes a controlled change and checks the result on other examples.

The common failure is premature generalisation: one correction quietly changes every future run.

Route by meaning, not wording

The same sentence can belong in different layers.

The words do not determine the destination. Scope, provenance and intended effect do.

Start with logical boundaries

You do not need four databases. One store can preserve the distinction if every item carries enough metadata and the application enforces the rules.

For each item, record:

1. its type;
2. who or what it applies to;
3. where it came from;
4. who may change it;
5. when it expires or becomes invalid;
6. how a change will be tested.

Then test the boundaries with real failure cases:

• Does campaign state leak into unrelated work?
• Does current knowledge supersede old rules?
• Can a skill change be regression-tested and rolled back?
• Can the team trace a remembered correction to the change it caused?
• Does that change improve held-out work without breaking other cases?

This is more informative than measuring whether retrieval returned a relevant paragraph. Retrieval can succeed while routing is wrong.

The separation has a cost

More types mean more metadata, routing logic and ownership. A short-lived internal assistant may not need this structure. A single prompt and a small document set may be enough.

The separation becomes useful when an agent works across customers, projects or long periods; when feedback can change future behaviour; or when a wrong instruction has a meaningful cost.

Even then, start with the smallest implementation that exposes the boundaries. Separate schemas and update policies may be enough. Add stores or services only when access, scale, security or operational ownership requires them.

What this post does not claim

This is an engineering taxonomy, not a universal definition of memory. It does not prescribe a vector database, knowledge graph or memory product. It also does not show that four layers automatically make an agent better.

At Replai, the separation gives us a clearer way to reason about specialised creative agents. The result still has to earn its complexity through fewer leaks, safer updates, reproducible skill changes and better accepted work.

An agent should know more than what to retrieve. The surrounding system must know what kind of information it is, where it applies and what process is allowed to change it.

Sources

1. Cognitive Architectures for Language Agents
2. Memory overview — LangChain Docs
3. Effective context engineering for AI agents
4. Equipping agents for the real world with Agent Skills