Apple Machine Learning Research1d agoenergy 51

Weighted State Souping for Efficient Long-Context Retrieval

This paper was accepted at the Workshop on Personalized, Aligned, Long-term Memory for AI Systems (PALM) at NeurIPS 2026. State space models (SSMs) combine fast inference and fixed state size with competitive language-modeling performance. Recently, state souping has been proposed as a procedure whereby an SSM independently encodes chunks of tokens like documents, tool definitions, or memory records. The resulting state encodings may be cached and recombined to cheaply construct a context for downstream queries. However, both state souping and vanilla SSMs deteriorate on long, crowded...

read the original at Apple Machine Learning Research →
This source publishes only a summary in its feed, so the full article lives at the outlet. The link above and below goes to the original.

more from the stream

Evaluating the Evaluators: Investigating LLM Judges for Personalized Writing Style Assessment (Apple Machine Learning Research) On the Importance of Gating: Memorization vs. In-Context Learning in State Space Models (Apple Machine Learning Research) Attention to Mamba: A Recipe for Cross-Architecture Distillation (Apple Machine Learning Research) ← back to upsy.si