Q-learning, DQN, actor-critic, DDPG, TD3, PPO, SAC and A3C: the update rules, losses, and a full A3C implementation, with what each method fixed.
Deep Dives
Worked examples: real bugs in real projects and the reasoning behind each fix, with AI in the loop.
Featured Posts
Integrate TanStack Router, TanStack Query, and a Shadcn sidebar into an Astro app with a co-located routing structure.
Testing whether a time series is a martingale using Black-Scholes simulation, permutation entropy, and Welch's PSD method.
Build an accessible, animated category accordion with native HTML5 details/summary, CSS transitions, and no JavaScript library.
The math behind diffusion models: Chapman-Kolmogorov, Kramers-Moyal, Ito's lemma, and the forward/reverse SDEs.
How GRPO turns next-token prediction into a reinforcement learning objective, derived step by step from the Transformer forward pass.
Keep IndexedDB and Pglite out of SSR with TanStack Router's ClientOnly, avoiding the classic null database instance error.
A design sketch for combining Titan's surprise-based memory with Transformer² SVD fine-tuning, with the math worked out and no experiments run yet.
How SakanaAI's Transformer² adapts LLM weights at inference time via SVD fine-tuning, RL, and expert weight combination, mapped from paper to code.
The state-space model, priors, and Gibbs sampling steps behind CausalImpact, plus how to turn posterior samples into a causal effect estimate.
All Categories
5Generative AI
Latest Posts
Testing whether a time series is a martingale using Black-Scholes simulation, permutation entropy, and Welch's PSD method.
The math behind diffusion models: Chapman-Kolmogorov, Kramers-Moyal, Ito's lemma, and the forward/reverse SDEs.
How GRPO turns next-token prediction into a reinforcement learning objective, derived step by step from the Transformer forward pass.
A design sketch for combining Titan's surprise-based memory with Transformer² SVD fine-tuning, with the math worked out and no experiments run yet.
How SakanaAI's Transformer² adapts LLM weights at inference time via SVD fine-tuning, RL, and expert weight combination, mapped from paper to code.
4Front-end Development
Latest Posts
Keep IndexedDB and Pglite out of SSR with TanStack Router's ClientOnly, avoiding the classic null database instance error.
Build an accessible, animated category accordion with native HTML5 details/summary, CSS transitions, and no JavaScript library.
Split client state and server state cleanly with Jotai and TanStack Query, and drop Redux Toolkit's boilerplate.
Integrate TanStack Router, TanStack Query, and a Shadcn sidebar into an Astro app with a co-located routing structure.
3Data Science
Latest Posts
Q-learning, DQN, actor-critic, DDPG, TD3, PPO, SAC and A3C: the update rules, losses, and a full A3C implementation, with what each method fixed.
The state-space model, priors, and Gibbs sampling steps behind CausalImpact, plus how to turn posterior samples into a causal effect estimate.
The Gamma distribution's PDF, its recursive and special-case identities, and how Erlang, Chi-squared, Exponential, Weibull and Poisson derive from it.
1AI Tools
Latest Posts
A practical guide to wiring Serena into Claude Code and tracking token usage and cost with ccusage, including current commands.
1Backend
Latest Posts
Use a Google service account to sign and verify M2M JWTs with custom claims, with a working NestJS example.
1Web Development
Latest Posts
Generate a sitemap.xml from your TanStack Start route tree at build time and wire up privacy-friendly analytics.