Can Large Language Models `Hyper-Thread'?

Engineering notes

Engineering notes will be added by the aipentium editorial team.

Chinese explanation / 中文解读

中文解读待补充：本站会优先为大语言模型、生成式AI、ChatGPT相关技术、计算机视觉、深度学习等高价值论文补充中文说明。

Original abstract

Large language models generate tokens sequentially, but can they execute multiple tasks concurrently while forming each token? Broader attention allocation may provide a mechanism for such task concurrency. Existing approaches to scaling inference primarily rely on longer generations, more samples, or additional verification stages, while attention dispersion is often treated as a signal of interference or error. Task concurrency within serial generation therefore remains underexplored. We propose the \textbf{Model Hyper-Threading Hypothesis} and evaluate its predictions using multiple coordinated tasks that share state within the same problem. We design three conditions---\base{}, \coarse{}, and \fine{}---and evaluate their benefits and costs using accuracy, output-token distributions, and attention metrics. On an AIME 2025 development set, \fine{} achieves the highest accuracy. Relative to \coarse{}, its typical output length is similar and it is shorter on most problems, while exhibiting greater attention dispersion and higher task-relevant coverage, albeit with a heavier output-length tail. Within-step concurrency and its causal mechanism still require direct tests. Our results show that more dispersed attention can coexist with higher accuracy, providing preliminary behavioral and correlational evidence for the hyper-threading hypothesis. These findings motivate a shift in perspective on inference scaling from ``generating more tokens'' toward ``having each generation step carry more tasks,'' pointing to a new avenue for improving reasoning performance.

Links and sources

Comments