What attention is, and what it isn't
How does an LLM connect a word to the words that came before it?
Take "The trophy didn't fit in the suitcase because it was too big". Here "it" is the trophy. To read "it", an LLM has to look back at the earlier words. Attention is the part that does the looking back.
