ds4
DeepSeek 4 Flash and PRO local inference engine for Metal, CUDA, and ROCm. It helps run capable open-weight models on consumer hardware without relying on a cloud service.
Share on XLicense: MIT
Overview
DwarfStar is a local inference engine for running a small set of capable open-weight models on consumer hardware. It addresses the need to use large language models without a remote service by supporting Metal, CUDA, and ROCm, and by including SSD streaming for larger models. The project is built around a narrow set of models and uses GGUF files generated by the project itself.
Key features
- Runs DeepSeek V4 Flash and PRO, GLM, and Qwen models locally
- Supports Metal, NVIDIA CUDA, and ROCm hardware targets
- Uses project-generated GGUF files and SSD streaming for larger models
- Includes integration testing for loading, prompt rendering, tool calls, KV state, HTTP server, and coding agent
Best for
This fits people who want to run open-weight models on their own Mac, CUDA, or ROCm system instead of a remote service. Pick it when the model set and hardware match the project’s supported targets and you want a narrow, local inference setup.
- Upstream
- antirez/ds4
- Fork on GitHub
- Guo-astro/ds4
- Upstream stars
- 23k
- Category
- AI agents and LLM tools
- Language
- C
- License
- MIT
- Forked
- 2026-10-03
- Sync status
- In syncLast synced 2026-10-04
More in AI agents and LLM tools
Strata lets you run a large local AI model on a gaming PC so work stays on your hardware. It helps with chats, code work, image input, and app integration without needing a server.
Forked 2026-10-03Last synced 2026-10-04License: MITAI agents and LLM toolsGitHub
Strands SDK helps teams build and run production AI agents in Python or TypeScript with any model and cloud, without writing a custom agent loop.
Forked 2026-10-02Last synced 2026-10-04License: Apache-2.0AI agents and LLM toolsGitHub
Trailblaze lets coding agents drive real devices and web apps with natural-language steps, then saves sessions as replayable trails for CI tests. It keeps flows repeatable without LLMs at replay time.
Forked 2026-10-02Last synced 2026-10-04License: Apache-2.0AI agents and LLM toolsGitHub
Rungic turns a compatible Android phone into a Linux desktop and pairs it with an assistant that can open apps, click, type, and complete tasks for you.
Forked 2026-10-01Last synced 2026-10-04No license declaredAI agents and LLM toolsGitHub