Noureddine RAMDI / Dot: an offline Electron desktop app for local LLM inference and document QA

Created Sat, 23 May 2026 20:41:14 +0000 Modified Sat, 23 May 2026 20:41:27 +0000
956 Words
github-stars javascript electron local-ai rag faiss llm

alexpinel/Dot

Dot offers a practical way to run large language models and augmented document search entirely on your desktop, without cloud or API dependencies. It’s packaged as an Electron app that bundles local LLM inference, Retrieval Augmented Generation (RAG), and Text-To-Speech (TTS), targeting non-programmers with prebuilt binaries for Apple Silicon and Windows. This approach sidesteps the usual complexity of setting up LLMs locally or relying on cloud APIs.

What Dot does and how it works

At its core, Dot is an offline-capable desktop application built with Electron, designed to bring local LLM inference and document question answering (QA) to users who may not be comfortable with command lines or cloud setups. It supports loading documents in various formats like PDF, DOCX, PPTX, XLSX, and others, then allows users to query them interactively on-device.

The architecture integrates several heavyweight machine learning components under the hood. For document retrieval, it uses FAISS, a fast vector similarity search library, to index document embeddings locally. The embeddings and vector store enable Retrieval Augmented Generation, which combines document retrieval with large language model inference to provide context-aware answers.

For the LLM backend, Dot relies on llama.cpp and Hugging Face models, running inference locally through these frameworks. It defaults to the Phi-3.5 model but is designed to work offline without requiring API keys or cloud services. Langchain is utilized to handle conversation chains and manage interaction flow, providing a conversational interface to the underlying LLM and document retrieval.

Text-To-Speech capabilities are integrated as well, allowing the app to vocalize answers, making it a more complete assistant experience.

The choice of Electron allows Dot to present a polished desktop user interface that is cross-platform (Apple Silicon and Windows currently), while packaging the complex ML logic in a way that is accessible without developer expertise.

technical strengths and design tradeoffs

What distinguishes Dot is the convergence of local LLM inference, RAG, and TTS into a single, offline-capable desktop application aimed at non-technical users. Many local LLM projects target developers comfortable with command lines or require cloud API keys; Dot lowers that barrier significantly.

The integration of FAISS for vector storage within the Electron app is noteworthy because FAISS is a C++ library typically used in server or backend environments. Packaging it inside an Electron desktop app means the developers had to bridge native code with the Node.js/Electron environment efficiently.

Using llama.cpp plus Hugging Face models for local inference is a strong choice to maintain offline functionality. These components allow running fairly capable models on local machines with Apple Silicon or Windows, though naturally the performance and model size are constrained by local hardware.

Langchain handles conversation chains, which is a solid choice to structure the interaction logic, but it adds a layer of dependency and complexity. The tradeoff here is between flexibility and the overhead of managing chain states and prompts locally.

Electron itself is a tradeoff: it provides a familiar UI framework and cross-platform consistency but comes with a significant resource footprint compared to native apps or lighter frameworks like Tauri or Rust-based GUIs. However, the choice favors developer DX and packaging convenience.

Overall, the codebase balances complexity and usability. It bundles multiple advanced ML tools in a single app without requiring the user to manage dependencies, API keys, or cloud infrastructure. The tradeoff is that local hardware limits model size and inference speed, and Electron’s resource usage is non-trivial.

Quick start

To use Dot:

  • Visit the Dot website to download the application for Apple Silicon or Windows.

For developers:

# Clone the repository
$ https://github.com/alexpinel/Dot.git

# Install Node.js dependencies
npm install
# If issues occur, try
npm install --force

# Move to the 'aadotllm' subdirectory and install dependencies there as well
cd aadotllm
npm install

This setup readies the project for local development or customization. The prebuilt binaries allow most users to run the app without any of this.

verdict

Dot is a solid option if you want to experiment with local LLM inference combined with document QA and TTS in a fully offline environment. Its Electron-based packaging and prebuilt binaries make it accessible to non-developers, removing the friction of API keys or cloud services.

The main limitation is hardware-dependent performance: local models like Phi-3.5 via llama.cpp are smaller and less capable than cloud-hosted giants. Electron’s resource footprint might be a drawback for those seeking lightweight apps.

Developers interested in local-first AI assistants will appreciate the integration of FAISS, Langchain, and local LLM inference in a desktop app. It’s a good reference for how to bundle complex ML tooling in a user-friendly package.

If your use case demands the highest model accuracy or large-scale multi-user deployments, cloud-based solutions still have the edge. But for privacy-conscious users or offline needs, Dot is worth exploring.

→ GitHub Repo: alexpinel/Dot ⭐ 1,910 · JavaScript