yep, show me how they're linked
The query (Q), key (K), and value (V) matrices all come from the same input tokens. Each token is multiplied by three different learned weight matrices (Wq, Wk, Wv) to create Q, K, and V.
Attention(Q, K, V) = softmax( (Q × K^T) / sqrt(dk) ) × V
So the Q and K vectors decide how much each token should attend to every other token (that's the similarity score), and then those scores are used to weight the V vectors. That's the direct connection.
In short: Q and K control "who looks at who", and V is what actually gets passed forward based on that attention. All three are derived from the exact same input embeddings, just transformed differently.
Have your own question?
Ask Quuik →