nanochat
A minimal, hackable codebase for training your own small ChatGPT-style model on a single GPU node. It covers tokenizing, pretraining, finetuning, evaluation and chat inference at low cost.
Pick a small issue you can reproduce, then fix it with your AI coding agent.
Share on XLicense: MIT
Overview
nanochat is a minimal, hackable harness for training your own small ChatGPT-style language model on a single GPU node. It covers tokenization, pretraining, finetuning, evaluation and inference, and lets you chat with the result through a simple command line interface. One setting, depth, which is the number of transformer layers, drives the other hyperparameters automatically. The README says a GPT-2 level model can be trained for about 48 dollars in roughly two hours on an 8XH100 node.
Key features
- Covers tokenization, pretraining, finetuning, evaluation and inference
- Runs on a single GPU node with minimal code
- One depth setting derives the other hyperparameters
- Chat with the trained model from a simple CLI
Best for
Good for learners and researchers who want to understand and modify the full LLM training pipeline at low cost. It maintains a leaderboard for GPT-2 speedruns.
- Upstream
- karpathy/nanochat
- Fork on GitHub
- Guo-astro/nanochat
- Upstream stars
- 59k
- Category
- AI agents and LLM tools
- Language
- Python
- License
- MIT
- Forked
- 2025-10-14
- Sync status
- In syncLast synced 2026-10-22
More in AI agents and LLM tools
/last30days is an AI-agent skill for researching recent discussions across social platforms, video, news, markets, and the web. It brings scattered signals together in a synthesized brief for trend research and preparation.
Forked 2026-10-22Last synced 2026-10-22License: MITAI agents and LLM toolsGitHub
A portable C99 inference engine for running Kimi K3 on a CPU without a GPU or machine-learning framework. It streams the large checkpoint from disk to support inference with limited RAM, for batch text generation or text chat.
Forked 2026-10-21Last synced 2026-10-22License: Apache-2.0AI agents and LLM toolsGitHub
Magnitude runs open models locally and tunes kernels for your hardware. Use it to connect local models to supported agents or an OpenAI-compatible client.
Forked 2026-10-19Last synced 2026-10-22License: Apache-2.0AI agents and LLM toolsGitHub
A collection of AI research skills and guides that helps researchers with idea evaluation, literature research, paper writing, and pre-submission review.
Forked 2026-10-18Last synced 2026-10-22Custom license (see repository)AI agents and LLM toolsGitHub