MIRA / Motivation

Expressive memory without pole-geometry bias.

A signal-processing perspective on the architecture of recurrent language models.

Language as a temporal signal

MIRA is a signal-processing-inspired large language model architecture. It treats token sequences as high-dimensional temporal signals and recurrent memory as an infinite-impulse-response (IIR) system. In signal processing, poles describe how a system retains, decays and oscillates in response to an input.

Restricting poles to the real axis restricts the memory modes a recurrent architecture can represent. MIRA is motivated by expressive memory without pole-geometry bias: a lattice–ladder realization admits non-real poles and input-conditioned dynamics.

Coupled forward signals and backward memory states propagate information through the lattice. A selective ladder readout combines the stages, while associative state transitions support parallel GPU execution. The aim is to bring the expressive dynamics of signal processing to efficient language modeling.

Why pole geometry matters

An IIR system describes how past inputs continue to affect the current output. Its poles govern the timescales and oscillatory behavior of those responses. Restricting the pole geometry imposes an architectural preference on the temporal modes that memory can represent.

MIRA addresses this restriction with a parameterized recurrent system that admits non-real poles. The rotation controls the coupling of forward signals and backward memory, while the radius controls their attenuation.

Why a lattice–ladder realization

The lattice supplies a structured cascade of coupled state transitions. The ladder combines the states through an input-conditioned readout. Together, they provide a signal-processing realization of expressive recurrent memory.

The same transitions admit associative composition over token segments, enabling a chunkwise parallel schedule on GPUs. The architecture therefore connects the choice of memory dynamics with an efficient implementation.

Read the dynamics equations →

Explore the CUDA execution schedule →