Back to all articles
Microsoft Agent Framework Azure AI Foundry Agentic AI Mem0 Memory

Your AI Agent Has Goldfish Brain — Two Ways to Fix It in Microsoft Agent Framework

A practical comparison of two memory architectures for Microsoft Agent Framework 1.0 — Mem0 as a tool the agent calls, and Azure AI Foundry Memory as a transparent context provider.

May 16, 2026 6 min read
Share

Imagine in-house counsel tells an AI agent that the firm never accepts auto-renewal clauses and caps liability at 12 months of fees. Twenty minutes later, the agent misses an auto-renewal on page 14 and waves through an uncapped liability clause.

That is a goldfish agent. The fix is memory — and Microsoft Agent Framework 1.0 supports two very different shapes of it.

Every Conversation Starts From Zero#

Most AI agent tutorials hide a dirty secret: the model only sees what is stuffed into the current prompt. When the session ends, the chat history dies with it. Fine for a hello-world chatbot — broken for contract reviewers, legal research assistants, tutors, or anything requiring continuity.

A useful memory layer needs three things:

  • Persistence — client positions survive the session
  • Relevance — the right clause stance is recalled at the right time
  • Isolation — user A's playbook never leaks into user B's review

Reference implementation for both approaches: github.com/imniteen/Microsoft-Agent-Framework-MyAgents. The src/mem0 and src/azure-ai-foundry-memory folders implement the two architectures discussed below.

The difference is who decides when to read and write — the agent itself, or the framework around it.

Option 1: Mem0 as a Tool the Agent Calls#

Mem0 is a hosted memory platform that handles embedding, deduplication, and semantic search given text and a user_id. You wrap the Mem0 client in a small class and expose its methods as function tools.

class MemoryTools:
    def __init__(self, client: MemoryClient, user_id: str):
        self.client = client
        self.user_id = user_id
        self._user_filter = {"AND": [{"user_id": user_id}]}
 
    def save_memory(self, information: Annotated[str, Field(...)]) -> str:
        """Save important user information to long-term memory."""
        self.client.add(
            [{"role": "user", "content": information}],
            user_id=self.user_id,
        )
        return f"Saved to memory: {information}"
 
    def search_memory(self, query: Annotated[str, Field(...)]) -> str:
        """Search long-term memory for relevant information about the user."""
        results = self.client.search(query, filters=self._user_filter)
        memories = _extract_memories(results)
        if not memories:
            return "No relevant memories found."
        return "\n".join(f"- {m['memory']}" for m in memories)

Register the tools on the agent:

agent = Agent(
    client=client,
    name="PersonalAssistant",
    instructions=AGENT_INSTRUCTIONS,
    tools=[
        memory_tools.save_memory,
        memory_tools.search_memory,
        memory_tools.get_all_memories,
    ],
)

The catch: the instructions block must explicitly tell the agent when to read and write.

1. At the START of every conversation turn, call 'search_memory' with a query
   relevant to the user's message to recall any useful context.
2. When the user shares standing positions, redlines, or playbook rules,
   call 'save_memory' to store them.
3. When the user asks "what do you know about my preferences?", call 'get_all_memories'.
4. Weave remembered playbook rules naturally into your reviews.

Without explicit workflow instructions, the model may ignore the memory tools entirely. You pay for memory in tokens, latency, and drift risk as the prompt grows.

Option 2: Foundry Memory as a Transparent Context Provider#

Foundry Memory takes the opposite approach — no tools, no workflow prompt. A context provider wraps the agent, pulling relevant memories before each turn and extracting new ones afterward.

Step one — create a memory store with a policy:

options = MemoryStoreDefaultOptions(
    chat_summary_enabled=False,
    user_profile_enabled=True,
    user_profile_details=(
        "Remember the user's standing client positions, contract clauses "
        "they routinely reject, preferred jurisdictions and governing law, "
        "liability caps, and risk tolerance. "
        "Avoid sensitive data such as client identifying information, "
        "billing records, or privileged matter notes."
    ),
)
memory_store = await project_client.beta.memory_stores.create(
    name=memory_store_name,
    description="Contract review playbook memory",
    definition=MemoryStoreDefaultDefinition(
        chat_model=os.environ["FOUNDRY_MODEL"],
        embedding_model=os.environ["AZURE_OPENAI_EMBEDDING_MODEL"],
        options=options,
    ),
)

You tell Foundry what to remember and what to avoid. Extraction is policy-driven, not prompt-driven.

Step two — wire the provider onto the agent:

memory_provider = FoundryMemoryProvider(
    project_client=project_client,
    memory_store_name=memory_store.name,
    scope=MEMORY_USER_SCOPE,
    update_delay=0,
)
 
async with Agent(
    name="PersonalAssistant",
    client=client,
    instructions="""You are a contract review assistant with long-term memory.
Memories from previous matters are automatically provided to you as context.
Use them to apply the user's playbook to every review.""",
    context_providers=[
        memory_provider,
        InMemoryHistoryProvider(load_messages=False),
    ],
) as agent:
    ...

Notice there is no tools=[...] parameter, and the instruction block is roughly half the length of the Mem0 version with no workflow rules.

Step three — inspect stored memories:

res = await project_client.beta.memory_stores.search_memories(
    name=memory_store.name,
    scope=MEMORY_USER_SCOPE,
)
for m in res.memories:
    print(m.memory_item.content)

After running the demo, entries about rejecting auto-renewal clauses and capping liability at 12 months of fees appear automatically — extracted by the provider, not saved on instruction.

Side-by-Side#

AspectMem0Foundry Memory
How memory happensAgent calls toolsProvider auto-handles
Prompt complexityHigh — you write the workflowLow — "memories are in your context"
StorageMem0 cloudAzure AI Foundry store
AuthMem0 API keyAzure AD (DefaultAzureCredential)
EmbeddingsManaged by Mem0You deploy your own
VisibilityEvery read/write in the traceHidden inside the provider
Best forFine-grained control, non-Azure stacksAzure shops wanting managed memory

Trade-offs#

Mem0 offers transparency at the cost of prompt weight. Every turn pays in tokens and latency, and the agent can drift if instructions slip. But for a contract reviewer that needs to justify why a clause was flagged or missed, that visible trace is valuable.

Foundry Memory delivers a managed service at the cost of visibility. Stores are still beta with no portal page today — inspection is SDK-only. Explaining why the agent applied a past playbook rule becomes harder.

There is no universal winner. Pick the trade-off that matches your team and your regulator. The instruction block is the tell — a numbered workflow means tools; "memories are in your context" means a provider.

Takeaways#

  1. Memory is not a bolt-on feature; it changes how you write the agent.
  2. Tool-based memory (Mem0) gives control and visibility — useful when defending decisions to partners or regulators.
  3. Provider-based memory (Foundry) gives a managed service; the framework absorbs the policy at the cost of inspection ease.
  4. Microsoft Agent Framework 1.0 supports both shapes cleanly — choose based on your stack.

Try It Yourself#

git clone https://github.com/imniteen/Microsoft-Agent-Framework-MyAgents.git
cd Microsoft-Agent-Framework-MyAgents
 
# Mem0 path
cd src/mem0
python -m venv .venv && .venv\Scripts\activate
pip install -r requirements.txt
python main.py
 
# Foundry Memory path
cd ..\azure-ai-foundry-memory
python -m venv .venv && .venv\Scripts\activate
pip install -r requirements.txt
az login
python main.py

Test it: tell the assistant that the client never accepts auto-renewal clauses and caps liability at 12 months of fees, then paste a vendor agreement containing both and ask whether it is safe to sign. The first reply tells you whether your agent has memory — or a goldfish brain.

An agent that forgets a client's redlines is like a junior associate who skipped the playbook. Build the memory in.

NB

Written by

Niteen Badgujar

AI Engineer specializing in Agentic AI, LLMs, and production-grade machine learning systems on Azure. Writing to make complex AI concepts accessible and actionable.

GitHubMediumMore about me