how-to-train-your-gpt
A beginner-friendly guide to building and training a GPT from scratch. It turns a difficult ML topic into a commented, runnable path for learning how language models work.
Pick a small issue you can reproduce, then fix it with your AI coding agent.
Share on XLicense: MIT
Overview
This project is a long, commented guide to building a modern language model from scratch. It explains the core ideas behind GPT, including tokenization, embeddings, position handling, attention, training, and inference, with runnable Python examples on each step. It is intended for people who want to understand how large language models work from the inside, without skipping the internal mechanics or relying only on APIs.
Key features
- Build a GPT step by step from scratch
- Learn tokenization, embeddings, attention, and training
- Follow runnable Python code and chapter explanations
- Understand model tradeoffs and inference behavior
Best for
This is best for Python developers, students, and engineers who want to understand how GPT works from the inside rather than just calling an API. It is a good fit when you want a long, commented guide and are willing to work through examples.
- Upstream
- raiyanyahya/how-to-train-your-gpt
- Fork on GitHub
- Guo-astro/how-to-train-your-gpt
- Upstream stars
- 4.2k
- Category
- AI agents and LLM tools
- Language
- Jupyter Notebook
- License
- MIT
- Forked
- 2026-10-05
- Sync status
- In syncLast synced 2026-11-08
More in AI agents and LLM tools
/last30days is an AI-agent skill for researching recent discussions across social platforms, video, news, markets, and the web. It brings scattered signals together in a synthesized brief for trend research and preparation.
Forked 2026-10-22Last synced 2026-11-08License: MITAI agents and LLM toolsGitHub
A portable C99 inference engine for running Kimi K3 on a CPU without a GPU or machine-learning framework. It streams the large checkpoint from disk to support inference with limited RAM, for batch text generation or text chat.
Forked 2026-10-21Last synced 2026-11-08License: Apache-2.0AI agents and LLM toolsGitHub
LayerFS gives AI agents writable workspaces over shared history, so they can work in parallel without copying the base. Use it to checkpoint, branch, rewind, and merge agent changes.
Forked 2026-10-21Last synced 2026-11-08License: MITAI agents and LLM toolsGitHub
Magnitude runs open models locally and tunes kernels for your hardware. Use it to connect local models to supported agents or an OpenAI-compatible client.
Forked 2026-10-19Last synced 2026-11-08License: Apache-2.0AI agents and LLM toolsGitHub