Breaking Down Transformers: The Core of Attention as Low-Rank Approximation
Why attention? How does it work?
Posts by Tag
transformer
attention layer
Breaking Down Transformers: The Core of Attention as Low-Rank Approximation
Why attention? How does it work?
low-rank approximation
Breaking Down Transformers: The Core of Attention as Low-Rank Approximation
Why attention? How does it work?
cosine similarity
Breaking Down Transformers: The Core of Attention as Low-Rank Approximation
Why attention? How does it work?
reconstruction error
Breaking Down Transformers: The Core of Attention as Low-Rank Approximation
Why attention? How does it work?