Back to Reading Room

LA BIBLIOTHÈQUE · READING NOTE

Improving Language Models by Retrieving from Trillions of Tokens

Borgeaud et al.

To Read · 想看

AIMemoryPapers

RETRO, External Memory, Retrieval, Non-Parametric Memory

RETRO 将超大规模文本数据库作为显式外部记忆,在语言模型预测时检索相关文本并通过 Cross-Attention 使用,为 Retrieval-Enhanced LM 提供重要范式。