ds4
DeepSeek 4 Flash and PRO local inference engine for Metal, CUDA, and ROCm. It helps run capable open-weight models on consumer hardware without relying on a cloud service.
Share on XLicense: MIT
Overview
DwarfStar is a local inference engine for running a small set of capable open-weight models on consumer hardware. It addresses the need to use large language models without a remote service by supporting Metal, CUDA, and ROCm, and by including SSD streaming for larger models. The project is built around a narrow set of models and uses GGUF files generated by the project itself.
Key features
- Runs DeepSeek V4 Flash and PRO, GLM, and Qwen models locally
- Supports Metal, NVIDIA CUDA, and ROCm hardware targets
- Uses project-generated GGUF files and SSD streaming for larger models
- Includes integration testing for loading, prompt rendering, tool calls, KV state, HTTP server, and coding agent
Best for
This fits people who want to run open-weight models on their own Mac, CUDA, or ROCm system instead of a remote service. Pick it when the model set and hardware match the project’s supported targets and you want a narrow, local inference setup.
- Upstream
- antirez/ds4
- Fork on GitHub
- Guo-astro/ds4
- Upstream stars
- 23k
- Category
- AI agents and LLM tools
- License
- MIT
- Forked
- 2026-10-03
- Sync status
- In syncLast synced 2026-10-06
More in AI agents and LLM tools
A curated library of DESIGN.md files for AI coding agents, helping teams reuse real design system patterns to generate matching UI from a reference design.
Forked 2026-10-05Last synced 2026-10-06License: MITAI agents and LLM toolsGitHub
A library of marketing skills for AI coding agents, covering CRO, copywriting, SEO, analytics, and growth engineering. Add the skills to a project so agents can apply marketing workflows to their work.
Forked 2026-10-05Last synced 2026-10-06License: MITAI agents and LLM toolsGitHub
An agent skill that turns natural language and system sources into editable architecture diagrams. It helps teams keep system views aligned with code, infrastructure, and APIs as architecture changes.
Forked 2026-10-05Last synced 2026-10-06License: MITAI agents and LLM toolsGitHub
Strata lets you run a large local AI model on a gaming PC so work stays on your hardware. It helps with chats, code work, image input, and app integration without needing a server.
Forked 2026-10-03Last synced 2026-10-06License: MITAI agents and LLM toolsGitHub