OpenKB tackles a common pain point in knowledge management with LLMs: the typical retrieval-augmented generation (RAG) approach treats each query statelessly, rediscovering information on the fly. Instead, OpenKB builds a compiled, persistent wiki-style knowledge base that accumulates and cross-references insights over time. It uses a vectorless retrieval mechanism under the hood, relying on a hierarchical tree index called PageIndex to handle long documents efficiently. This approach offers a fresh angle on LLM-powered knowledge bases, trading off the complexity and cost of vector embeddings for a structured, incremental wiki.
what OpenKB does and how it works
OpenKB is a Python CLI tool designed to transform raw documents—PDFs, markdown, and other text—into a structured, persistent knowledge base. Unlike typical RAG systems that embed documents into vector spaces for similarity search, OpenKB uses PageIndex, a vectorless long-document retrieval technique that indexes documents into hierarchical trees. This allows reasoning-based retrieval through tree traversal instead of nearest-neighbor search.
At a high level, OpenKB processes short documents by converting them fully into markdown with wikilinks and summaries. For longer PDFs (20 pages or more, configurable via pageindex_threshold), it applies PageIndex to create a hierarchical tree index that segments the document into logical chunks for smarter retrieval. These chunks then link to concept pages and summaries, forming an interlinked wiki.
The resulting knowledge base is a plain markdown wiki compatible with Obsidian and other markdown-based note-taking apps. It supports multi-LLM backends through LiteLLM, allowing users to plug in different LLM providers for compilation and chat features.
Additional features include a watch mode for live auto-compilation on file changes, interactive chat sessions with persisted context to maintain state across conversations, and lint checks that highlight contradictions or knowledge gaps in the wiki content.
the architecture and tradeoffs behind OpenKB’s vectorless retrieval
What sets OpenKB apart is its departure from the typical vector embeddings approach popular in RAG workflows. Instead of embedding every query and document chunk into a vector space, it builds a persistent compiled wiki maintained by LLMs, where information accumulates and cross-references evolve.
The core of this is PageIndex, which indexes long documents into hierarchical tree structures. This indexing enables reasoning-based retrieval that respects document structure and context, rather than relying solely on vector similarity. This is particularly useful for large PDFs, where naive chunking and embedding can lose semantic hierarchy.
The tradeoff here is clear: vectorless retrieval avoids the complexity, cost, and sometimes unpredictable behavior of vector embeddings but requires careful upfront compilation and maintenance of the wiki. It also means the knowledge base is not as flexible for ad-hoc queries outside compiled knowledge.
OpenKB’s codebase is surprisingly clean for a project dealing with complex LLM orchestration and document processing. The use of Python and its CLI interface keeps dependencies minimal. Integration with LiteLLM abstracts LLM providers, enhancing flexibility. The architecture favors convention over configuration, making it approachable for users who want a persistent knowledge store rather than a query-time retrieval system.
quick start with OpenKB
Install
pip install openkb
Other install options
Latest from GitHub:
pip install git+https://github.com/VectifyAI/OpenKB.gitInstall from source (editable, for development):
git clone https://github.com/VectifyAI/OpenKB.git cd OpenKB pip install -e .
This straightforward installation gets you the CLI ready to process documents and build your wiki. The README and documentation provide details on commands for compiling wikis, running chat sessions, and using watch mode.
verdict
OpenKB is a solid choice for developers and knowledge workers looking to build a persistent, interlinked knowledge base with LLMs without relying on vector embeddings. Its vectorless retrieval via PageIndex is particularly appealing if you work with large documents like lengthy PDFs and want to preserve their structure in retrieval.
The tradeoff is the upfront cost of compilation and the less flexible ad-hoc querying compared to vector-based RAG. But if your workflows benefit from a stable, evolving knowledge wiki with cross-document synthesis and multi-LLM support, OpenKB is worth exploring.
The code is clean and accessible, making it a viable starting point for further customization or integration into larger knowledge management systems. Just be aware that the approach requires maintaining the wiki incrementally, which might not fit all use cases where immediate query freshness or breadth is paramount.
Overall, OpenKB offers a practical and interesting alternative to the vector embedding paradigm, showing that structuring knowledge as a living, LLM-maintained wiki is a viable path for persistent knowledge retrieval.
Related Articles
- Pathway LLM App: unified pipelines for scalable retrieval-augmented generation and AI search — Pathway LLM App provides integrated pipelines for scalable RAG and AI search, combining vector and full-text indexing wi
- Navigating free-tier LLM APIs with the awesome-free-llm-apis catalog — A curated catalog of free-tier LLM APIs compatible with OpenAI SDK, detailing rate limits, model specs, and providers to
→ GitHub Repo: VectifyAI/OpenKB ⭐ 1,253 · Python