Noureddine RAMDI / Memary: Recursive Knowledge Graph Memory for Autonomous AI Agents

Created Mon, 04 May 2026 10:23:02 +0000 Modified Sat, 23 May 2026 20:41:27 +0000
1025 Words
github-stars ai knowledge-graph memory-management python llm streamlit

kingjulio8238/Memary

Memary offers a practical approach to the persistent memory problem in autonomous AI agents. Instead of flat logs or simple vector stores, it uses knowledge graphs to represent and query agent memories, enabling multi-hop reasoning and recursive retrieval. This reduces the latency and irrelevance issues that plague full-graph queries as memory grows.

What memary does and its architecture

Memary is a Python-based open-source project designed to provide a memory layer for autonomous AI agents by emulating human memory structures. It uses knowledge graphs as the underlying data model to store and retrieve memories, capturing relationships between entities and events in an interconnected graph.

The architecture revolves around a key class named ChatAgent, which handles memory streaming (continuous capture of interactions), entity knowledge storage, and managing tools that agents might use. The system is built on top of LlamaIndex, a library for interacting with graph data structures and indexing.

Memary supports two primary knowledge graph backends: FalkorDB and Neo4j. These graph databases store the agent’s memory as nodes and edges, representing entities and their relationships. Agent memories are stored as graphs that can be queried recursively to find relevant context efficiently.

The system integrates with large language model APIs such as OpenAI and Perplexity to query and reason over the stored knowledge graphs. This allows agents to perform multi-hop reasoning, connecting pieces of information across the graph to answer complex queries.

A Streamlit dashboard is included for visualizing the current memory state, entity relationships, and agent interactions. This visualization aids in debugging and understanding memory flow.

Key features include:

  • Recursive retrieval: identifies key entities in a query and constructs subgraphs with a maximum depth of 2 around these entities.
  • Multi-hop reasoning: joins these subgraphs to form a context relevant to the query.
  • Multi-agent memory isolation: supports multiple agents with isolated knowledge graphs, enabling separate memory contexts.
  • Automatic memory capture: agent interactions and user preferences are automatically stored.
  • Persona-based agents: user and system personas can be customized to shape agent behavior.

The recursive retrieval approach and memory isolation as technical strengths

Memary’s main technical strength lies in its recursive retrieval method for querying knowledge graphs. Instead of querying the entire graph, which grows large and slow, it isolates key entities mentioned in the query and fetches subgraphs centered on those entities up to two hops away. These subgraphs are then combined to answer the query.

This design reduces latency since it avoids expensive full-graph traversals. It also improves relevance by focusing only on the parts of the graph pertinent to the query. The multi-hop reasoning capability allows the agent to connect distant but related pieces of information, enhancing the quality of responses.

The memory isolation for multi-agent setups is another important strength. Each agent can have its own graph, preventing memory leakage or cross-contamination. For FalkorDB users, the system supports switching between multiple graphs representing different agents. This isolation is crucial for applications where multiple personas or agents operate independently.

Under the hood, the ChatAgent class manages streaming of memories into the graph, entity extraction, and tool management. The codebase leverages LlamaIndex for graph operations, which provides an abstraction layer for indexing and querying.

Tradeoffs include the complexity of managing knowledge graphs and dependencies on external API keys and graph databases. The recursive retrieval strategy may also miss some context if the subgraph depth or entity extraction is imperfect. However, these tradeoffs are typical for graph-based memory systems aiming for efficiency and relevance.

The code is surprisingly clean and modular, with clear separation between memory management, agent logic, and API integration. The Streamlit dashboard adds a developer-friendly interface to monitor and debug agent memory.

Quick start with memary

Setting up Memary involves a few steps to configure API keys and environment variables, then running the Streamlit app for interaction.

  1. Install Memary via pip (ensure you are using the appropriate Python version).

  2. Set up your .env file with required API keys for OpenAI, Perplexity, Google Maps, and Alpha Vantage. Configure database URLs and passwords for FalkorDB or Neo4j.

  3. Fetch API credentials for each service as per their instructions. FalkorDB and Neo4j offer free instances for testing.

  4. Customize the user persona by editing streamlit_app/data/user_persona.txt using the provided template.

  5. Optionally, update the system persona in streamlit_app/data/system_persona.txt.

  6. For FalkorDB users, multi-graph support allows managing multiple agents’ memory graphs.

  7. Run the Streamlit app:

cd streamlit_app
streamlit run app.py

This launches a dashboard where you can interact with the ChatAgent, observe memory states, and test queries.

Verdict: who should consider memary?

Memary is a solid choice if you are building AI agents that require persistent, context-rich memory and want to avoid the pitfalls of token limits or flat vector stores. Its recursive retrieval using knowledge graphs balances efficiency and relevance, making it suitable for agents that need multi-hop reasoning over complex memories.

It fits best for developers comfortable with Python, graph databases, and managing several API integrations. The multi-agent memory isolation is a plus for applications with multiple personas.

Limitations include the need to manage external services (OpenAI, Perplexity, FalkorDB/Neo4j) and potential complexity in setting up and fine-tuning memory retrieval parameters. Also, the recursive retrieval depth is capped, which may miss some extended context.

Overall, Memary is a practical, well-structured memory layer for experimental and production AI agents that benefit from knowledge graph representations. It’s worth exploring if you want to build autonomous agents with better long-term memory and reasoning capabilities.

→ GitHub Repo: kingjulio8238/Memary ⭐ 2,604 · Jupyter Notebook