Abstract

Manta’s agents reason over a per-tenant knowledge graph: an in-memory RDF store that runs inference to plan and take actions on a business’s behalf. For an agent like that, “memory” is not a transcript to search; it is context that must be correct on every turn. The dominant approach, embedding the chat history and retrieving nearest neighbors, fails that bar in three specific ways: a hard constraint that isn’t lexically similar to the current message gets dropped; text can’t be reasoned over, only re-read; and chunks have no lifecycle, so contradictions pile up.

Our answer is to make a memory a first-class node in the graph, not a row in a vector index. A memory is a typed RDF entity with a kind, a scope, an importance, a source, and (when it represents something structured like a target or a budget) a subclass carrying strongly-typed properties. Because memories obey the same ontology, the same shape constraints, and the same rule engine as the business data beside them, the graph can infer over them: a stored performance target is joined against the live KPI to derive whether it is being met, with no model call in the loop. Memory stops being storage and becomes inference.

1. Why an agent needs more than a transcript

An agent that only answers questions can get away with fuzzy memory. An agent that takes actions on a business cannot. The moment software can spend money, pause a campaign, or message a customer, “mostly remembered the context” is not good enough.

The industry default is retrieval-augmented generation over an embedding store: chunk the history, embed it, and each turn pull back the k nearest neighbors of the user’s latest message. It’s a fine tool for “what did we discuss last week.” It is the wrong primitive for a system that acts, for three reasons that compound:

1. Similarity is not relevance. A hard rule like “never spend more than $40k/month on Meta” is relevant to every budgeting decision, forever. But it is not semantically close to “draft me three headlines,” so nearest-neighbor recall silently drops it at the exact moment it matters. Relevance here is structural (“this is a spending decision”), not lexical.

2. Text can’t be reasoned over. A budget stored as a sentence is inert. To ask “are we over budget right now?” the system must re-read the sentence, re-parse the number, and re-join it to current spend on every turn, with no guarantee that two turns agree.

3. Chunks have no lifecycle. They don’t supersede each other, don’t expire, don’t carry a scope or an owner, and can’t be audited. Change a target from ROAS 3.0 to 2.5 and the old chunk sits in the index contradicting the new one.

None of this is news to anyone running retrieval in anger. Mature vector stacks already bolt structure back on: metadata filters to pin a type = constraint, hybrid dense-plus-keyword search, rerankers. A newer generation of agent-memory systems goes further, with temporal knowledge graphs such as Zep / Graphiti adding validity intervals and supersession, and extraction layers such as Mem0 adding a lifecycle to remembered facts. That’s the right instinct, and it’s the one we take to its conclusion. The question this article answers is what you get when structure isn’t a filter bolted onto retrieval but the primary representation: a memory as a typed node in the same graph, under the same rules, as the business data it’s about. The differentiator isn’t “we use a graph.” It’s that the graph can infer over memory and join it against live facts, not merely store and fetch it.

An embedding index is a good recall mechanism and a nonexistent reasoning one. An agent that acts needs both.

2. Memory as knowledge, not text

We already run, per customer, a knowledge graph: an in-memory RDF store and reasoner that materializes that business’s world of campaigns, ad accounts, orders, KPIs, and (in our recruiting vertical) roles and candidates. It then runs a rule program over that graph to derive new facts, validate them against a schema, and drive the agent’s planning loop.

So we put memory in that graph itself, not in a separate vector service alongside it. It lives as nodes that speak the same ontology as everything else.

Once a memory is a graph citizen, everything the graph does to a fact it does to a memory: constrain it with a shape, infer new facts from it, join it against live business data, scope it to a person or a company, expire it, supersede it, audit it. None of that is available to a text chunk in an index.

3. The shape of a memory

A memory carries a small, deliberate set of properties. Two typing dimensions sit at the center.

Kind is a closed set of four: preference (how someone likes to work), fact (durable context), objective (a target to pursue), and context (cross-conversation background, captured automatically). Kind is coarse on purpose: it drives how a memory is grouped when the agent reads it.

Subclass is the open, extensible dimension. Beyond its kind, a memory can be an instance of a domain class that is a subclass of the base :Memory type: PerformanceTargetMemory, BudgetConstraintMemory, BrandVoiceMemory, and so on. A subclass brings its own strongly-typed, shape-validated properties. This is the move that turns a memory from prose into a record you can compute with.

Alongside those, every memory carries the connective tissue: an importance (an integer 1–5, from Disabled through Critical), a required source recording where it came from, a confidence (1 for anything a human entered), an optional expiry, and (once it’s ever replaced) a pointer to the memory that superseded it. Importance is the primary recall dial: the top tiers are always in context; the middle tiers are fetched on demand; the bottom is excluded entirely.

One consequence sets up §7: the base type holds the shared properties, and each subclass declares only its own. A single rule (§7) makes every subclass instance count as a :Memory, so validation, recall, and supersession work on a new subclass without any new code. Deriving a status from live data is the part that isn’t automatic: that’s a rule someone writes, per subclass.

4. Scope, precedence, and isolation

A memory’s scope is one of user, team, or company, and it is not cosmetic: it decides who sees the memory, who may edit it, and how conflicts resolve.

Scope is backed by stakeholder links, not just a label: a personal memory is tied to a specific person, a team memory to one or more teams, a company memory to everyone on the tenant. A scoped read is one query with a branch per scope.

Writes are authorized against the same stakeholder links: a regular user can curate their own memories, while shared company and team memories are operator-gated, and you cannot supersede a memory you don’t own.

When memories from different scopes disagree, the agent applies the more specific one (personal over team over company) and flags the conflict when it looks accidental rather than deliberate. That precedence rule lives in the agent’s instructions, so contradictions resolve deterministically instead of by vibes.

Scope governs sharing inside a tenant. Isolation governs the boundary between tenants: each customer’s graph is a physically separate store, addressed by the identity in the request. There is no shared memory table filtered by a tenant_id column that a query bug could leak across; two customers’ memories never sit in the same store. Every memory also carries its required source and an owner, so it is always attributable and always contained.

5. Getting memories in: capture without ceremony

Memories arrive three ways, and all three converge on the same validated RDF. What differs is only the recorded source.

• The agent writes them. The model has a structured memory tool. It doesn’t save a sentence; it fills in a typed memory (kind, scope, importance, an optional subclass and its properties, an optional expiry), and the write path validates and stores it. The set of available subclasses is described to the model, so it knows a “budget cap” is a BudgetConstraintMemory with a platform and an amount, not free text.

• The system extracts them. Nobody should have to say “remember this.” After a conversation accumulates enough new turns, a background process assembles the unprocessed tail, pulls the already-known and topically-related memories so it won’t duplicate them, and hands the bundle to a durable background workflow that does the extraction and writes the results (typically context-kind memories) back into the graph. A periodic sweep catches idle conversations too. Extraction runs off the hot path, so it never delays a reply, and it’s retryable: a failure re-runs instead of dropping context.

• People curate them. An operator can author, edit, and organize memories directly, and a short onboarding flow seeds a new tenant with its first objective, fact, and preference so the agent starts with a spine of high-value context on day one.

Underneath, a write is a sanitized graph update: an insert for new memories, or a precise diff for edits. The diff uses careful three-state semantics so a partial edit never clobbers a field it didn’t touch, and every value is escaped so nothing user-supplied can break out of its slot.

6. Getting memories out: recall that doesn’t flood context

Recall is the harder half. Every turn, the agent has to pick what to show itself: enough to be useful, not so much that a hard constraint drowns. The strategy is layered.

• Pre-load the high-value core. Before each response, the agent reads the tenant’s memories fresh (deliberately uncached, because cross-conversation memory only works if something written in one chat is visible in another moments later) and keeps the high-priority ones (the top importance tiers, above a confidence floor). Those are grouped into labeled blocks (objectives, preferences, context, and one block per subclass), so the model sees them organized, each tagged with an identifier it can cite if it later wants to revise that exact memory.

• Rank by similarity inside the eligible set. Within those blocks, ordering is by a built-in semantic-similarity score against the recent conversation, so the most contextually apt memories come first. This is the one place embeddings earn their keep: a ranking signal over a structurally-filtered set, never the gate. Structure decides what’s eligible; similarity decides what’s on top.

• Share company memory through the graph, nt per user. High-value company memories are injected via the graph’s live context (§8) so every user shares one copy, rather than duplicating a company constraint into everyone’s window.

• Fetch the long tail on demand. Mid- and low-importance memories aren’t pre-loaded; the agent retrieves them with an explicit search tool when the task calls for it. This tail fetch is ordinary similarity retrieval, and it carries the same “relevant-but-dissimilar might be missed” risk we pin on RAG in §1. But it’s confined to memories we’ve already deemed non-critical: the important stuff is guaranteed present by structure, and only the low-stakes remainder is left to similarity. The agent is told this, so it knows a Critical constraint is already in context while a minor preference must be looked up.

The result: a hard constraint is present because it’s important, not because it happened to resemble the last message.

7. The payoff: subclasses that infer

Some memories aren’t facts to recall; they’re standing conditions to check against reality. A performance target only means something next to the current KPI, a budget cap only next to current spend. Because these memories are typed subclasses sitting in the same graph as the live numbers, a rule can join them to those numbers and keep their status current. An index can’t do this at all.

It starts with one rule. Rather than switch on the full RDFS ruleset, we assert subclass membership directly, and this is what keeps the design open-ended.

The body treats the ontology’s own subclass edges as data. To make that hold for multi-level hierarchies (a subclass of a subclass), we also close subClassOf transitively with one more rule, so a class anywhere beneath :Memory still qualifies.

With those two rules, a new class defined anywhere under :Memory is immediately a first-class memory: validation, recall, and supersession already match it, with no code change. The reasoning is the part that isn’t free: a subclass that should carry a live status needs its own join rule.

A replaced memory should disappear from normal use but stay in the graph for audit, so we express “not yet replaced” as an inferred class.

A performance-target memory stores a metric and a goal. Because a store holds exactly one tenant’s world (§4), there is a single :Company node, and the rule matches the active target against that company’s live KPI to derive a status.

A budget-constraint memory works the same way against spend, deriving within_budget / near_limit / over_budget. (Metrics where lower is better, like cost-per-acquisition, simply flip the comparison; the direction of “good” is encoded in the rule, so the model never has to remember it.) The rule language, stratified negation, and FILTER are all standard RDFox reasoning constructs.

The user said “keep ROAS above 3” once, months ago, and that became a PerformanceTargetMemory. Each time the KPIs refresh from the data pipeline, the graph re-derives whether the target is met, with no model call and no re-reading of the original sentence. By the time the agent loads context, targetStatus = below is already sitting there as a fact. The memory itself isn’t clever; a rule outside it re-ran the join the moment the numbers moved.

Worked example: a budget cap, three weeks on. In March a user says “don’t let Meta spend go over $40k this month.” The agent mints a BudgetConstraintMemory: platform meta, amount 40000, company scope, Critical importance. Nothing else happens. Three weeks later a different user, in a different conversation, asks “can we scale the spring campaign?” Before that chat emits a token, the budget memory is already in context, and because spend refreshed to $34k overnight, the rule has re-derived near_limit. The agent opens its reply knowing the cap, who set it, and that the account sits at 85% of it. Each of those is a fact it reads, not one it reconstructs.

8. Live context: memory that keeps re-evaluating

The graph maintains a materialized bundle of what a fresh conversation should start with (among them the KPI snapshot, the connected accounts, the subscription, and the high-value company memories), assembled by rules rather than by application code. A company memory joins the bundle when it is active (not superseded), company-scoped, in the top importance tiers, and either non-expiring or not-yet-expired. Expiry is compared against a clock node inside the graph (a scheduled tick advances it), so it is enforced by the rules themselves rather than by application code remembering to check a date at read time. A guardrail rule caps how large the bundle may grow, so context can’t quietly bloat.

At runtime the two halves meet. A performance-target memory enters the bundle because it’s active, company-scoped, and important, and it arrives already carrying its derived status, because the inference ran on the same graph. The agent opens a brand-new conversation already knowing both the goal and whether it’s being met.

Two properties make this trustworthy. The store performs incremental materialization: when the pipeline updates spend, only the affected derived facts recompute, so a status is never stale relative to its inputs. And writes are serialized per tenant against a single-writer store, so concurrent memory writes and pipeline updates don’t corrupt one another.

9. Conflict, supersession, and audit

Because goals and facts change, new memories contradict earlier ones, so replacement is a first-class, auditable event.

Supersession is a deliberate dual pointer. The primary write goes on the old memory: “superseded by X”. The inverse, “X supersedes this”, is derived.

The asymmetry is intentional: the derived forward pointer vanishes if the old memory is ever deleted, but the “superseded by” fact persists on the old node, so the newer memory stays authoritative even after cleanup; history isn’t corrupted by a delete.

Three mechanisms catch conflicts:

1. Automatic dedup. Re-stating a memory with the same label in the same scope updates it instead of piling up a duplicate.

2. Semantic conflict detection. On a substantive new memory, a background model is shown the existing same-scope memories and asked which the newcomer would replace. Auto-captured context resolves silently; anything a human might have meant to keep triggers an ask-before-overwrite: the write pauses and the agent confirms whether to replace or keep both. We never silently clobber a human’s memory.

3. Explicit replacement. When the agent already knows what it’s replacing, it says so and detection is skipped.

Replaced memories are hidden from normal reads but stay in the graph, restorable by an operator (and the restore refuses if a still-live memory would then contradict it). Combined with the required source and timestamps, every memory’s origin and lineage is reconstructable. We keep provenance deliberately lightweight (source plus supersession chain) rather than adopting a full formal provenance ontology, which would be a lot of triples per node to answer a question we mostly ask as “where did this come from, and what did it replace.”

10. Trade-offs and limitations

• Eventual, not transactional, freshness. Derived statuses are near-real-time, not point-in-time consistent with the source systems. For steering, that’s the right trade; a number that must be exact-to-the-second should be read from the source, not the graph.

• Importance is a human (or model) judgment, and recall leans on it hard. If a genuinely critical memory is filed as Medium, it won’t pre-load. We mitigate with sensible defaults and by letting the model set importance from context, but the dial can be set wrong.

• We still depend on embeddings, just not for correctness. Similarity ranking can bury a relevant-but-dissimilar memory within an eligible block. Because the block is already importance-filtered, the blast radius is ordering, not omission, but ranking quality still matters.

• Subclass proliferation is a real cost. The open subclass model is powerful and tempting; a sprawl of near-duplicate memory types would erode the shared-machinery benefit. Keeping the ontology small and curated is ongoing work, not a solved problem.

• Extraction can be wrong. An LLM deciding what’s worth remembering will sometimes capture noise or miss signal. Confidence scoring, dedup, and the ask-before-overwrite guard contain the damage, but auto-captured memory is assistive, not authoritative.

• Rules must be written for incrementality. Deriving status by joining memories against live data is only cheap because the store maintains it incrementally; a carelessly shaped rule can force expensive recomputation. Rule design is a discipline, not an afterthought.

11. Where this goes

The pattern generalizes past marketing. The memory engine (the subclass rule, the active/supersession lifecycle, the live-context injection) is domain-agnostic; only the subclass ontology changes per vertical. Our recruiting product defines its own memory types (hiring criteria, candidate assessments, sourcing preferences) on the exact same base, and inherits validation, recall, and supersession for free. Domain-specific inference (the recruiting analog of “is this target being met”) is then layered on with the same rule patterns.

Memory also sits upstream of action. The same context that answers a question shapes the plans the agent proposes, and lets it flag when a plan would breach a stored constraint. As the agent starts proposing its own memory subclasses, and as more standing conditions become derived signals, the gap between what the agent remembers and what it knows to be true right now keeps closing. A memory that reasons is just knowledge, kept current.