ds4
DeepSeek 4 Flash and PRO local inference engine for Metal, CUDA, and ROCm. It helps run capable open-weight models on consumer hardware without relying on a cloud service.
Pick a small issue you can reproduce, then fix it with your AI coding agent.
Share on XLicense: MIT
Overview
DwarfStar is a local inference engine for running a small set of capable open-weight models on consumer hardware. It addresses the need to use large language models without a remote service by supporting Metal, CUDA, and ROCm, and by including SSD streaming for larger models. The project is built around a narrow set of models and uses GGUF files generated by the project itself.
Key features
- Runs DeepSeek V4 Flash and PRO, GLM, and Qwen models locally
- Supports Metal, NVIDIA CUDA, and ROCm hardware targets
- Uses project-generated GGUF files and SSD streaming for larger models
- Includes integration testing for loading, prompt rendering, tool calls, KV state, HTTP server, and coding agent
Best for
This fits people who want to run open-weight models on their own Mac, CUDA, or ROCm system instead of a remote service. Pick it when the model set and hardware match the project’s supported targets and you want a narrow, local inference setup.
- Upstream
- antirez/ds4
- Fork on GitHub
- Guo-astro/ds4
- Upstream stars
- 24k
- Category
- AI agents and LLM tools
- Language
- C
- License
- MIT
- Forked
- 2026-10-03
- Sync status
- Updated by last syncLast synced 2026-10-24
More in AI agents and LLM tools
/last30days is an AI-agent skill for researching recent discussions across social platforms, video, news, markets, and the web. It brings scattered signals together in a synthesized brief for trend research and preparation.
Forked 2026-10-22Last synced 2026-10-24License: MITAI agents and LLM toolsGitHub
A portable C99 inference engine for running Kimi K3 on a CPU without a GPU or machine-learning framework. It streams the large checkpoint from disk to support inference with limited RAM, for batch text generation or text chat.
Forked 2026-10-21Last synced 2026-10-24License: Apache-2.0AI agents and LLM toolsGitHub
LayerFS gives AI agents writable workspaces over shared history, so they can work in parallel without copying the base. Use it to checkpoint, branch, rewind, and merge agent changes.
Forked 2026-10-21Last synced 2026-10-24License: MITAI agents and LLM toolsGitHub
Magnitude runs open models locally and tunes kernels for your hardware. Use it to connect local models to supported agents or an OpenAI-compatible client.
Forked 2026-10-19Last synced 2026-10-24License: Apache-2.0AI agents and LLM toolsGitHub