Table of Contents

• Why integrate Titan and Transformer²?
• Review
  • Titan’s surprise-driven memory update
  • Transformer² Singular Value Fine-tuning
• A design for integrating Titan and Transformer²
  • • State, action, and transitions
  • Policy and reward
  • Policy gradient update
• Conclusion
• What would validate this
• Related work

Why integrate Titan and Transformer²?

Think about how you recall a meeting: you forget most of the back-and-forth but keep the decisions. Human memory compresses aggressively, keeping the salient parts and discarding the rest. That is the property this post tries to borrow for a memory module design.

The idea below is a design sketch, not a validated result. I have not run the experiments (no stable GPU access at the time of writing), so treat the following as a hypothesis with the math worked out, not a benchmarked claim.

The plan is to use Singular Value Decomposition (SVD) inside a Titan-style memory module so the model keeps the dominant patterns in its parameters and discards the rest, the same tradeoff a compressed memory makes.

Using SVD in a Titan-style memory module should, in principle:

• Compress information: SVD keeps the dominant singular directions of a weight update and drops the rest, similar to how a compressed memory keeps only the salient trace.
• Improve efficiency: low-rank updates are cheaper to store and apply than full-rank ones.

---

Review

Titan’s surprise-driven memory update

Titan (“Titans: Learning to Memorize at Test Time”, linked under Related Work below) updates a long-term memory module based on “surprise”: how unexpected new information is relative to what the memory already predicts. For an input token $x_t$, Titan projects it into key-value pairs:

$$k_t = x_tW_K, \quad v_t = x_tW_V,$$

and defines an associative memory loss:

$$\ell(M_{t-1}; x_t) = \big\|M_{t-1}(k_t) - v_t\big\|_2^2.$$

Using this loss, Titan updates a “surprise” momentum $S_t$ with forgetting:

$$S_t = \eta_t S_{t-1} - \theta_t \nabla \ell(M_{t-1}; x_t),$$

and then updates the memory with a forgetting gate:

$$M_t = (1 - \alpha_t)M_{t-1} + S_t.$$

This lets Titan encode surprising information while a forgetting gate lets ordinary, expected inputs decay from memory.

---

Transformer² Singular Value Fine-tuning

Transformer² (“Transformer²: Self-Adaptive LLMs”, also linked below) fine-tunes a weight matrix by scaling its singular values rather than updating the full matrix. For a weight matrix $W \in \mathbb{R}^{d \times d}$, SVD gives:

$$W = U S V^{T},$$

where

• $U \in \mathbb{R}^{d \times r}$,
• $S \in \mathbb{R}^{r \times r}$,
• $V \in \mathbb{R}^{d \times r}$,

with $r$ the rank of the decomposition.

This low-rank factorization cuts the number of trainable parameters while keeping the dominant structure of the weight matrix. Applied to a Titan-style memory module, the memory parameters become the set

$$\theta_t = \{U_t, S_t, V_t\},$$

which evolve over time as the model trains.

---

A design for integrating Titan and Transformer²

Combining Titan’s memory update with Transformer²’s low-rank parameterization gives a framework where an RL policy adjusts low-rank memory factors based on reward. Here is how that combination works out mathematically.

State, action, and transitions

At time $t$, the state is:

$$s_t = \Big(x_t,\, M_t,\, \theta_t\Big),$$

where $\theta_t = \{U_t, S_t, V_t\}$ are the decomposed memory parameters.

The action $a_t$ adjusts these factors:

$$a_t = \big(\Delta U_t,\, \Delta S_t,\, \Delta V_t\big).$$

The transition updates the parameters:

$$\begin{aligned} U_{t+1} &= U_t + \Delta U_t, \\ S_{t+1} &= S_t + \Delta S_t, \\ V_{t+1} &= V_t + \Delta V_t, \end{aligned}$$

which implies

$$W_{t+1} = U_{t+1}S_{t+1}V_{t+1}^{T}.$$

The memory itself still updates via the surprise mechanism, applied with the next step’s decomposed parameters:

$$\begin{aligned} S_{t+1} &= \eta_{t+1} S_t - \theta_{t+1} \nabla \ell(M_t; x_{t+1}), \\ M_{t+1} &= (1-\alpha_{t+1})M_t + S_{t+1}. \end{aligned}$$

Policy and reward

A policy $\pi_{\phi}(a \mid s)$ decides how to adjust the low-rank factors given the current state. The objective is to maximize expected cumulative reward:

$$J(\phi) = \mathbb{E}_{\tau \sim \pi_{\phi}}\Bigg[\sum_{t=0}^{T} \gamma^t r_t\Bigg],$$

where $r_t$ is the reward at time $t$, reflecting how well the model’s memory helps it perform on the task at hand.

Policy gradient update

By the policy gradient theorem, the update for policy parameters $\phi$ is:

$$\nabla_{\phi} J(\phi) = \mathbb{E}_{\pi_{\phi}}\Bigg[\sum_{t=0}^{T} \nabla_{\phi} \log \pi_{\phi}(a_t|s_t) \cdot G_t\Bigg],$$

with

$$G_t = \sum_{t'=t}^{T} \gamma^{\,t'-t}r_{t'}.$$

The policy update rule becomes:

$$\phi \leftarrow \phi + \alpha \, \nabla_{\phi} \log \pi_{\phi}(a_t|s_t) \cdot G_t.$$

Actions sampled from this policy update the SVD factors, which is the mechanism that ties memory adjustments to long-term reward instead of a fixed learning rule.

---

Conclusion

Combining Titan’s surprise-based memory update with Transformer²’s low-rank SVD parameterization gives a model where memory adjustments are learned via RL rather than fixed by a hand-tuned decay schedule. The math above is self-consistent, but it is a design, not a result: nothing here has been trained or benchmarked.

What would validate this

The next step is running it: fine-tune a small model with this policy, and check whether the learned $(\Delta U, \Delta S, \Delta V)$ updates actually improve task performance over the fixed forgetting-gate baseline. That experiment has not been run yet due to lack of stable GPU access; if it happens, the results belong in a follow-up post.

Related work

Paper:

• Transformer²: Self-Adaptive LLMs
• Titans: Learning to Memorize at Test Time

Code: SakanaAI’s Transformer² GitHub Repository

Blog: SakanaAI’s Transformer² Blog Post

If you want to run an experiment like this yourself, the tooling below covers the data and observability layer for tracking training runs, and the newsletter is where the results of this one will ship if the GPU access comes through.