Research setting Independent
An LLM has to connect words across a sentence to make use of its context. A heat kernel maps how a small pulse spreads through a network. My study uses that mathematical probe to examine attention: which token positions form close groups, what links those groups, and how that organization changes across heads and layers.
Weaker connections
The pulse stays in its starting group longer.
1.4% reached the other group
Stronger connections
The pulse reaches the other group sooner.
16.9% reached the other group