With interactive terminal programs, timing is everything. When automating or integrating AI agents with these programs, the challenge isn’t just sending input—it’s knowing exactly when the terminal UI has settled after output changes. tui-use tackles this by intercepting every byte of terminal output through a PTY event stream, feeding it into a headless xterm emulator, and applying a debounce timer to detect when the screen stabilizes. This approach removes the need for arbitrary sleep or polling loops and makes automated interactions with TUIs far more reliable.
What tui-use does and how it works
tui-use is a TypeScript CLI tool designed to bridge the gap between AI agents and interactive terminal programs such as REPLs, debuggers, and text-based user interfaces (TUIs). It does this by spawning target programs inside pseudo-terminal (PTY) sessions and capturing their output byte-by-byte. Instead of a blind stream of bytes, tui-use feeds this output into a headless xterm emulator — essentially a terminal emulator running without a UI — which interprets control sequences, cursor movements, color changes, and screen state.
This emulation produces a full snapshot of the terminal screen at any point, allowing tui-use to track the exact visible state of the interactive program it controls. The key innovation here is a snapshot-based interaction model combined with smart wait semantics. Rather than guessing wait times or polling for output changes, tui-use uses a debounce timer on PTY render events to determine when the screen has stabilized. Only then does it signal the AI agent or controlling process that it’s safe to proceed with the next action.
Under the hood, tui-use runs a daemon process that manages these PTY sessions and headless terminal emulators. This daemon keeps sessions alive across multiple CLI invocations, allowing persistent interactions with long-running terminal programs. The architecture is designed to be extensible, with plugin support for AI agents like Claude Code and OpenAI Codex that can consume the terminal snapshots and drive interactions intelligently.
The stack is TypeScript-based, running on Node.js, leveraging npm for distribution. It integrates tightly with terminal emulation standards via the xterm.js library or similar, though the exact library isn’t explicitly detailed in the README.
The PTY event stream and smart wait: why it matters
The problem of automating terminal UIs is deceptively hard. Many tools rely on sending keystrokes and then waiting a fixed amount of time or polling output streams for changes. This approach is brittle and often leads to race conditions or wasted time.
What sets tui-use apart is its direct observation of PTY render events. Instead of polling or guessing, it intercepts every output byte, feeds it to the headless emulator, and watches the resulting screen state. It then applies a debounce timer — a short delay that resets whenever new output arrives — to detect when the output stops changing. This “screen stability” signal is crucial for reliable automation.
This method avoids the common pitfalls seen in tools like tmux, which multiplex terminal sessions but don’t provide semantic knowledge of screen changes. tui-use’s approach allows AI agents to wait for specific prompts, UI highlights, or inverse-video cues before proceeding, enabling smarter and more context-aware interactions.
The tradeoff is increased complexity and resource usage, as tui-use must maintain full terminal emulation state and run a daemon. However, this cost pays off in accuracy and robustness, especially for complex TUIs that update screen content dynamically.
The code quality appears solid: the repo is clean, well-organized, and leverages TypeScript typings for safety. The snapshot model encourages a clear separation between terminal state tracking and agent logic. Plugins for AI agents demonstrate the architecture’s extensibility.
Quick start
From npm (recommended):
npm install -g tui-use
From source:
git clone https://github.com/onesuper/tui-use.git
cd tui-use
npm install
npm run build
npm link
Install from this repo
Step 1: Open this repository in Codex
Start Codex with this repository as the working directory, or restart Codex if it was already open while you cloned or updated the repo.
Step 2: Open the plugin directory
codex
/plugins
Step 3: Install the plugin
Choose the tui-use local plugins marketplace, open tui-use, and select Install plugin.
Step 4: Start a new thread
Ask Codex to use tui-use, or explicitly invoke the installed plugin/skill from the prompt.
Install from self-hosted marketplace
Step 1: Add the marketplace
/plugin marketplace add onesuper/tui-use
Step 2: Install the plugin
/plugin install tui-use@tui-use
Step 3: Reload plugins
/reload-plugins
More agents coming soon…
verdict
tui-use addresses a real pain point for anyone automating or integrating AI agents with interactive terminal programs. Its snapshot-based model and smart wait semantics solve the tricky problem of knowing when a TUI is stable enough to respond.
That said, it’s not a lightweight tool — the daemon architecture and full terminal emulation add complexity and resource overhead. It’s most relevant for developers building AI integrations that require precise, reliable terminal interaction rather than simple scripting.
If you’re automating REPLs, debuggers, or complex TUIs and need robust synchronization beyond naive sleeps or polls, tui-use is worth exploring. For simpler use cases, traditional expect-style tools or tmux might suffice.
Overall, tui-use is a pragmatic, well-crafted tool that fills a niche in AI-agent-to-terminal automation, with a clean architecture and extensible plugin system. Worth understanding even if you don’t adopt it outright.
Related Articles
- elizaOS: a TypeScript monorepo for building and deploying AI agents — Explore elizaOS, a TypeScript monorepo for AI agents with CLI and web UI. Build and deploy agents fast or extend with pl
- OpenAI Codex CLI: local-first AI coding assistant with ChatGPT integration — OpenAI Codex CLI brings AI coding assistance local to your terminal, integrating with ChatGPT plans for powerful hybrid
- Inside Google Gemini CLI: a terminal-first AI agent with extensible Model Context Protocol — Google Gemini CLI is a TypeScript-based terminal AI agent offering direct Gemini model access, extensibility via MCP, an
→ GitHub Repo: onesuper/tui-use ⭐ 229 · TypeScript