Noureddine RAMDI / Elato-Local: a local voice AI platform bridging desktop and embedded IoT on Apple Silicon

Created Tue, 05 May 2026 13:37:39 +0000 Modified Sat, 23 May 2026 20:41:27 +0000
1009 Words
github-stars typescript tauri esp32 voice-ai local-llm embedded rust python

akdeb/open-toys

Elato-Local stands out by running a full voice AI stack locally on Apple Silicon, completely avoiding cloud dependencies. What’s most interesting under the hood is how it bridges a desktop app environment with embedded IoT devices — the ESP32-S3 microcontrollers. The desktop app bundles firmware images, flashes the ESP32-S3 hardware directly via USB, manages WiFi captive portal setup, and maintains a WebSocket connection for device communication. This kind of tightly integrated development flow is uncommon in typical web or desktop apps, making Elato-Local worth exploring for anyone working at the edge of desktop and embedded systems.

What Elato-Local does and its architecture

Elato-Local is a fully local voice AI platform designed for interactive toys, companions, and robots. It runs entirely on Apple Silicon without relying on cloud services, emphasizing privacy and subscription-free use. It combines several machine learning components:

  • Whisper Turbo for speech-to-text (ASR), enabling robust local transcription.
  • Qwen3-TTS and Chatterbox-turbo for text-to-speech synthesis.
  • MLX-community large language models (Qwen3, Llama, Mistral) for conversation and dialogue.

The desktop application is built with Tauri, React, and Rust. Tauri provides a lightweight wrapper around a webview UI (React) and handles system-level operations efficiently via Rust. This choice avoids the bloat of Electron while delivering native performance and tighter OS integration.

The embedded hardware side uses ESP32-S3 microcontrollers. These devices are flashed with firmware images bundled directly inside the desktop app. Communication between the desktop app and the ESP32 happens over WebSocket connections after an initial USB flashing and WiFi captive portal setup.

A Python 3.11 backend runtime manages the AI model lifecycle — downloading and caching models on first run, then serving them locally.

This architecture cleanly separates concerns:

  • Desktop app handles UI, firmware flashing, device setup, and WebSocket communication.
  • Embedded devices focus on audio capture, playback, and network transport.
  • Local AI runtime performs speech recognition, natural language understanding, and speech synthesis.

The engineering behind firmware flashing and local AI inference

The standout engineering aspect is the firmware flashing flow integrated into the Tauri desktop app. Unlike typical IoT workflows that require separate flashing tools or command-line utilities, Elato-Local bundles bootloader, partition tables, and firmware images as resources inside the app.

When a user connects an ESP32-S3 device via USB, the app flashes the firmware directly by invoking the flashing sequences programmatically. This reduces friction for developers and end-users by embedding this step into the app itself.

After flashing, the device boots into WiFi captive portal mode, allowing users to connect the ESP32 to a local network. The app detects this state and manages WebSocket reconnection automatically to establish a persistent communication channel.

This seamless transition from USB flashing to WiFi setup to WebSocket communication is a rare engineering achievement. It tightly couples desktop and embedded workflows, a space where many projects struggle due to tooling fragmentation.

On the AI side, the use of Whisper Turbo and MLX-community LLMs for local inference means the whole voice AI stack runs without cloud calls. The Python backend downloads models on first use, keeping the initial setup manageable. Voice cloning with Qwen3-TTS adds personalization capabilities.

The codebase balances complexity:

  • The Tauri app uses Rust for system integration and React for UI.
  • ESP32 firmware is managed in sync with the desktop app.
  • Python backend handles model inference and runtime environment.

The tradeoffs include restricting hardware support to Apple Silicon and ESP32-S3 devices. The local ML models require significant compute, justifying Apple Silicon’s capability but limiting wider hardware compatibility.

Here’s a snippet illustrating how the desktop app handles firmware flashing (simplified):

async function flashFirmware(devicePort: string) {
  const bootloaderPath = getResourcePath('bootloader.bin');
  const partitionPath = getResourcePath('partition-table.bin');
  const firmwarePath = getResourcePath('firmware.bin');

  await esp32Flasher.flash(devicePort, {
    bootloader: bootloaderPath,
    partitionTable: partitionPath,
    firmware: firmwarePath,
  });
}

This function abstracts away complex serial flashing commands into a single call within the app, improving developer and user experience.

Explore the project

The repository is organized with clear separation:

  • src-tauri contains the Rust backend for the desktop app.
  • src holds the React frontend UI.
  • Firmware images and flashing scripts are embedded within the desktop app resources.
  • The Python backend runtime lives in a separate directory, managing AI model downloads and serving.

The README provides detailed documentation on architecture, device setup, and usage considerations. Given the hardware-specific nature and local AI focus, reading through the docs is essential before attempting deployment.

For those interested in embedded IoT or local AI voice systems, the repo is a rich resource showcasing cross-domain integration.

Verdict

Elato-Local is a compelling example of bridging desktop app development with embedded IoT workflows for local voice AI. Its firmware flashing flow from a Tauri desktop app is rare and improves developer and user experience significantly.

The project is best suited for developers with Apple Silicon machines and ESP32-S3 hardware who want a fully offline, privacy-first voice AI platform. It is not a plug-and-play solution for general consumers but a solid foundation for building custom interactive toys or robots with local AI.

Limitations include hardware specificity and the complexity of managing multiple tech stacks (Rust, React, Python, embedded C). The compute demands restrict broader device compatibility.

Still, for its niche, Elato-Local offers a clean architecture, solid codebase, and a practical example of embedding firmware flashing into a desktop app — a pattern worth understanding if you work at the intersection of desktop and embedded systems.

→ GitHub Repo: akdeb/open-toys ⭐ 91 · TypeScript