Back to Reading Room

LA BIBLIOTHÈQUE · READING NOTE

Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context

Dai et al.

To Read · 想看

AIMemoryPapers

Transformer-XL, Working Memory, Recurrence, Long Context

通过 Segment-Level Recurrence 将前一段序列的隐藏状态复用于后一段,使 Transformer 能跨越固定 Context Window 建模更长期依赖。