uzu
uzu is an inference engine for running AI models in apps, helping developers keep inference on-device instead of relying on remote services. Use it to download a model and build chat features.
Pick a small issue you can reproduce, then fix it with your AI coding agent.
Share on XLicense: MIT
Overview
uzu is a Rust inference engine for running AI models inside applications rather than sending prompts to a remote inference service. It addresses the need for app-integrated inference with data kept on the device and without per-request inference charges. Developers configure an engine, select and download a model, then create a chat session and request replies. The project also provides language bindings for Python, TypeScript, and Swift, and supports unified memory on Apple devices.
Key features
- High-level API for configuring models and creating chat sessions
- Download models and track download progress
- Traceable computations to check results against source implementations
- Language bindings for Python, TypeScript, and Swift; unified memory support on Apple devices
Best for
Developers who want to integrate AI model inference and chat into an application. It is a fit when keeping inference in the app and avoiding remote inference charges are priorities.
- Upstream
- trymirai/uzu
- Fork on GitHub
- Guo-astro/uzu
- Upstream stars
- 3.2k
- Category
- AI agents and LLM tools
- Language
- Rust
- License
- MIT
- Forked
- 2026-10-09
- Sync status
- Updated by last syncLast synced 2026-11-08
More in AI agents and LLM tools
/last30days is an AI-agent skill for researching recent discussions across social platforms, video, news, markets, and the web. It brings scattered signals together in a synthesized brief for trend research and preparation.
Forked 2026-10-22Last synced 2026-11-08License: MITAI agents and LLM toolsGitHub
A portable C99 inference engine for running Kimi K3 on a CPU without a GPU or machine-learning framework. It streams the large checkpoint from disk to support inference with limited RAM, for batch text generation or text chat.
Forked 2026-10-21Last synced 2026-11-08License: Apache-2.0AI agents and LLM toolsGitHub
LayerFS gives AI agents writable workspaces over shared history, so they can work in parallel without copying the base. Use it to checkpoint, branch, rewind, and merge agent changes.
Forked 2026-10-21Last synced 2026-11-08License: MITAI agents and LLM toolsGitHub
Magnitude runs open models locally and tunes kernels for your hardware. Use it to connect local models to supported agents or an OpenAI-compatible client.
Forked 2026-10-19Last synced 2026-11-08License: Apache-2.0AI agents and LLM toolsGitHub