Signal processing · Recurrent dynamics · LLMs

A LLM architecture
inspired by digital signal processing

Expressive recurrent memory without pole-geometry bias. MIRA draws on signal processing to model token sequences with input-conditioned lattice–ladder dynamics and a dedicated CUDA implementation.

Read paper

MIRA whale logo with a constellation and colored stars

01 / Introduction

A language model with
lattice memory.

A token enters a cascade of coupled forward signals and backward memory states. Input-conditioned rotations and radii control how information propagates; a selective ladder readout gathers information across the stages.

Ut(k−1) = rt(k) ⊙ (cos θt(k) ⊙ Ut(k) − sin θt(k) ⊙ Vt−1(k−1))
Vt(k) = rt(k) ⊙ (sin θt(k) ⊙ Ut(k) + cos θt(k) ⊙ Vt−1(k−1))

The lattice order K, expanded channel width de and per-channel memory dm define a recurrent state whose size stays fixed as the sequence grows.

Read the equations and parameterization →

02 / Evaluation

Language modeling
and CUDA execution.

Evaluate the architecture through language-model comparisons and dedicated GPU-kernel measurements.

Results depend on model geometry, corpus, context length and hardware. The evaluation pages present each comparison with its conditions.

03 / Build

From a mixer
to your own LLM.

Use the existing pretrainer with YAML configuration, or import MIRA building blocks into your own PyTorch architecture.

shell
python -m pip install .
python
from mira_llm import MiraMixer, MiraBlock, MiraLLM

Contact

Cetus AI Lab

Math-inspired, efficient next-generation architecture for building foundation models.

roisincrtai [at] gmail [dot] com