Songlin Yang, Bailin Wang, Yu Zhang, Yikang Shen, Yoon Kim

Parallelizing Linear Transformers with the Delta Rule over Sequence Length presents a new algorithm designed to enhance the performance and scalability of linear transformers. While linear transformers and state-space models have emerged as potential alternatives to traditional transformers with softmax attention, they still face limitations, particularly in tasks that require in-context retrieval.

The paper focuses on improving DeltaNet, a more expressive variant of linear transformers that uses the delta rule instead of the additive update found in traditional linear transformers. DeltaNet has been shown to improve associative recall but is hindered by lack of parallelization over sequence length in training.

The authors introduce a hardware-efficient algorithm for training these models by leveraging a memory-efficient representation for computing products of Householder matrices, which significantly reduces memory consumption and computational overhead.

Using the algorithm, the authors train a 1.3B parameter model on 100B tokens and show that it outperforms existing linear-time baselines such as Mamba and GLA in terms of both perplexity and zero-shot performance on downstream tasks.

Additionally, the paper explores two hybrid models that combine DeltaNet layers with either sliding-window attention or global attention layers, showing that these hybrids can outperform strong transformer baselines. This work provides a promising step forward in making linear transformers more practical and efficient for large-scale applications, potentially paving the way for more scalable models in the future.

Shaoteng Liu, Haoqi Yuan, Minda Hu, Yanwei Li, Yukang Chen, Shu Liu, Zongqing Lu, Jiaya Jia

RL-GPT introduces an innovative integration of Large Language Models (LLMs) and Reinforcement Learning (RL), offering a new approach to solving embodied tasks, or tasks that involve interacting with an environment.

While LLMs excel at generating high-level plans and understanding language, they often fall short when it comes to executing these plans with precision in dynamic, real-world scenarios.

Existing methods usually tackle this by fine-tuning LLMs with RL or using hierarchical RL structures. However, the authors of RL-GPT take a different approach: they treat RL as a tool for optimising precision in task-specific low-level actions, rather than relying on it for the entire process.

The paper introduces a two-level hierarchical framework consisting of a slow agent and a fast agent, trained in a two-loop iteration with a third critic agent to provide feedback. The slow agent is responsible for analysing high-level actions that are suitable for coding, while the fast agent creates code-as-policy for the RL action space. Their results outperform both traditional RL methods and previous GPT-based approaches on the MineDojo benchmark, a Minecraft-based testbed for evaluating embodied tasks.

What makes this work particularly thought-provoking is its broader implications for the future of AI systems. It shows that a combination of methods can form a larger system, where LLMs are involved but language isn’t the only interface for interaction, capable of handling a wider range of tasks. It raises the question of what applications will emerge as these methods continue to improve.