Temporal-Spectral Hamiltonian Mixers for Efficient Long Sequence Modeling

<jats:p>We present Temporal Spectral Hamiltonian Mixer, a lightweight and memory-efficient architecture for modeling long-range dependencies in sequential data. Inspired by Hamiltonian dynamics, TSHM introduces a structured module that maintains stability and long-term coherence while remaining simple to implement, efficient to train, and can handle ten to over hundred thousand of sequence length without memory overflow. The design supports both parallelized training and constant-time streaming inference O (1), making it suitable for real-time and low-latency applications. Compared with Transformers and structured state-space models(SSM) such as S4, TSHM achieves a favorable balance between expressivity, computational cost, and engineering simplicity and are naturally bidirectional or sequential in all direction. It avoids specialized kernels, requires only dense linear and pointwise operations, and operates with minimal memory overhead. We provide a mathematical formulation, interpret the model through a Hamiltonian lens, analyze its computational complexity, and outline a reproducible experimental plan across diverse benchmarks in speech recognition, text classification, time-series forecasting, and sequential image classification. Across benchmarks — Long Range Area(LRA), sMNIST, CIFAR (1-D), long horizon forecast dataset and Google Speech Commands, TSHM consistently outperforms Transformer variants, it is second or sometime competitive with SSM (S4), and shows largest gains on time-series forecasting. Notably, TSHM processes raw 16,000-sample speech sequences achieving 91% accuracy, LRA Path-X (16000 sequence length) achieving 80.31% while standard Transformer and Recurrent Neural Network (RNN) baselines fail under the same conditions because of memory overflow or efficiency. Finally, to demonstrate TSHM’s real-world practicality, we deployed it as firmware on an ESP32-S3 voice-control device. The model runs at 1.1–1.4 ms per frame and completes 10 head-only Stochastic Gradient Descent (SGD) updates in 1.2 ms, enabling accurate, privacy-preserving on-device learning without cloud dependency. The code:https://github.com/Ding777/TSHM.git</jats:p>