Publications
Projects
Teaching
3
Theory of Scaling Laws for In-Context Regression - Depth, Width, Context and Time
We study in-context learning (ICL) of linear regression in a deep linear self-attention model, characterizing how performance depends …
Blake Bordelon
,
Mary Letey
,
Cengiz Pehlevan
PDF
Cite
DOI
Pretrain-Test Task Alignment Governs Generalization in In-Context Learning
In-context learning (ICL) is a central capability of Transformer models, but the structures in data that enable its emergence and …
Mary Letey
,
Jacob A Zavatone-Veth
,
Yue M. Lu
,
Cengiz Pehlevan
PDF
Cite
DOI
Cite
×