Recurrent Transformer New Architecture: RLT Enables Tokens to 'Continue Computing' Between Each Other

Beating AI News Flash: Princeton PhD student Yifan Zhang proposes the Recurrent Looped Transformer (RLT), adding an internal state to the Transformer that continuously propagates forward.

When generating the next Token, a standard Transformer primarily reads prior context through Attention and KV caching. In addition to looking at prior context, RLT also takes the internal state left over from computing the previous Token and passes it directly to the next Token for continued use.

You can think of it as adding a "thinking relay baton" to the model. After the first Token is computed, it passes the current state to the second; after the second updates, it passes it to the third. The number of network layers each Token runs through on its own does not change, but this state chain grows longer as the text gets longer. The paper calls this "infinite temporal depth," referring to the fact that this computation chain has no fixed upper length limit.

It is also somewhat different from the recently much-discussed Ouro and Astra-style recurrent architectures. That type of method mainly has the same Token run through the same network multiple rounds; RLT, by contrast, continuously passes internal state between different Tokens. The former is like checking a problem several more times before handing in the paper, while the latter is more like one runner finishing a leg and handing the progress directly to the next runner.