Computation and Language (cs.CL)

Apr 22, 2026Formal Sciences260423.0005

Convergent Evolution: How Different Language Models Learn Similar Number Representations

Deqing Fu, Tianyi Zhou, Mikhail Belkin, Vatsal Sharan +1 more

This paper reveals that different language model architectures (Transformers, RNNs, LSTMs) converge on learning similar periodic number representations with periods at 2, 5, and 10, despite being trained differently. The authors identify a two-tiered hierarchy of these features and explain when models learn geometrically separable representations useful for modular arithmetic.

Apr 22, 2026Formal Sciences260423.0007

Working Memory Constraints Scaffold Learning in Transformers under Data Scarcity

Pranava Madhyastha, Dagmar Adamcova

This paper integrates working memory constraints into Transformer models through cognitively-inspired attention mechanisms and shows that these constraints improve grammatical accuracy and human alignment when training data is limited. The results suggest that mimicking human cognitive limitations can serve as a useful inductive bias for learning robust language representations.

Apr 22, 2026Formal Sciences260423.0047

DialToM: A Theory of Mind Benchmark for Forecasting State-Driven Dialogue Trajectories

Neemesh Yadav, Palakorn Achananuparp, Jing Jiang, Ee-Peng Lim

DialToM is a benchmark that tests whether LLMs truly understand Theory of Mind by evaluating both mental state prediction and the ability to forecast dialogue trajectories from those states. The study reveals that most LLMs can identify mental states but fail to use this understanding to predict realistic social interactions.

Apr 20, 2026Formal Sciences260421.0046

GSQ: Highly-Accurate Low-Precision Scalar Quantization for LLMs via Gumbel-Softmax Sampling

Alireza Dadgarnia, Soroush Tabesh, Mahdi Nikdan, Michael Helcig +2 more

GSQ is a new scalar quantization method for large language models that uses Gumbel-Softmax relaxation to jointly optimize grid assignments and scales, achieving accuracy comparable to complex vector quantization methods while remaining compatible with existing inference kernels. It successfully quantizes models to 2-3 bits per parameter and scales to trillion-parameter mixture-of-experts models.

Computation and Language

Latest in cs.CL

Convergent Evolution: How Different Language Models Learn Similar Number Representations

Working Memory Constraints Scaffold Learning in Transformers under Data Scarcity

DialToM: A Theory of Mind Benchmark for Forecasting State-Driven Dialogue Trajectories

GSQ: Highly-Accurate Low-Precision Scalar Quantization for LLMs via Gumbel-Softmax Sampling

Follow