From Attention to Applications

What does each attention block change? Recall the additive positional encoding studied earlier: eᵢ + pᵢ. The eᵢ in the residual equation denotes the current embedding entering that sublayer. Each head uses its own Q/K/V projections. The diagram uses the existing message mᵢ and update Δeᵢ convention; normalization is omitted. Today we vary which positions can attend and which output embeddings the task uses.

What kinds of output could we request for this sentence? Keep the sentence fixed. Reveal one branch per advance; collect suggested answers before naming any architecture. Throughout the lecture, each displayed word is treated as one token for clarity; actual tokenization may split words. Punctuation is omitted from the token strips. Labels and continuations are authored teaching examples, not model predictions.

What kinds of output could we request for this sentence? Keep the sentence fixed. Reveal one branch per advance; collect suggested answers before naming any architecture. Throughout the lecture, each displayed word is treated as one token for clarity; actual tokenization may split words. Punctuation is omitted from the token strips. Labels and continuations are authored teaching examples, not model predictions.

What kinds of output could we request for this sentence? Keep the sentence fixed. Reveal one branch per advance; collect suggested answers before naming any architecture. Throughout the lecture, each displayed word is treated as one token for clarity; actual tokenization may split words. Punctuation is omitted from the token strips. Labels and continuations are authored teaching examples, not model predictions.

What kinds of output could we request for this sentence? Keep the sentence fixed. Reveal one branch per advance; collect suggested answers before naming any architecture. Throughout the lecture, each displayed word is treated as one token for clarity; actual tokenization may split words. Punctuation is omitted from the token strips. Labels and continuations are authored teaching examples, not model predictions.

Does a Transformer automatically return just one summary vector? Trace Raghav to e₁, then goes to e₂. Recall eᵢ → attention message mᵢ → update Δeᵢ → e′ᵢ = eᵢ + Δeᵢ → MLP and residual. We keep the token-position subscript as its embedding is updated; the diagram labels identify the stage. These final embeddings are not the attention messages.

What should we identify before naming a model family? Return to these questions whenever the task changes.

Which state predicts school after Raghav goes to? Trace the message, projection, residual addition and MLP. The subscript identifies the receiving input position: m₃ belongs to to; m₄ would belong to school only after school is supplied. For several heads, concatenate their messages before W_O. The figure shows the final block and omits normalisation; earlier blocks repeat the same kind of update. A decoder-only stack has causal self-attention and feed-forward sublayers; it does not have a separate encoder or source cross-attention. The term decoder names its generation role. Scores cover the vocabulary even though the drawing shows one selected word.

Can Raghav’s state use the later word Delhi? Read the first row, then the last row. Filled cells mean access is allowed, not that weights are equal.

What changes after we predict the next token? Trace each word to its input embedding eᵢ and final contextualised embedding eᵢ. Within a block, attention retrieves mᵢ; the output projection gives Δeᵢ; add it to eᵢ, then apply the MLP and its residual. The amber output is the last supplied position, which feeds the vocabulary head. Input embeddings include positions. The input and updated-embedding labels distinguish the stages without introducing a superscript. The displayed continuation is an example sequence, not a model run. Under causal attention, appending a word does not change earlier positions’ final embeddings for a fixed model in evaluation. The new position reads the available prefix; caching can reuse earlier computations. Multiple head messages are concatenated before the output projection. Normalisation is omitted from the diagram.

Is the vocabulary head another stack of hidden layers? Follow the dense connections through the MLP. Trace the residual bypass into the plus sign. The resulting updated e₁ is still eight numbers, ready for the next slide. Widths 8 and 12 are chosen for drawing, not a specific model. Normalization is omitted, and its exact placement depends on the architecture. The left vector already includes the attention update and its residual. The final e₁ denotes the updated embedding at position 1, not the embedding of the word to be generated. Multiple stacked blocks provide multiple hidden layers. Some model variants use more elaborate prediction heads, but they are not required here.

Are the candidate outputs limited to words in the sentence? Trace all eight input features to the output neurons. Point to bicycle and banana: these are valid vocabulary entries even though they are unlikely continuations here. The highlighted goes neuron has the largest illustrative logit. Vocabulary size 50,000 and embedding width eight are illustrative, not specifications of a named model. W has shape 8 × 50,000, b has shape 1 × 50,000 and the output has shape 1 × 50,000. The vocabulary is fixed by the tokenizer; ##ing is an illustrative subword notation. The neuron list is a selection, not vocabulary order or a ranking. All scores are authored examples. The seven visible logits are [1, -2, 5.5, -1, 1.5, -3, 3]; each omitted entry has logit -8 for the numerical softmax example. Some models omit the bias. W here is distinct from the attention projection W_O.

Does softmax consider only the displayed tokens? Compare goes with the unrelated tokens. The omitted entries contribute to normalization even though they are not drawn. Trace the selected token into Raghav goes; its updated e₂ is read at the next step. The seven visible logits and 49,993 omitted logits of -8 define this authored distribution at temperature 1. Percentages are rounded to two decimal places; exact probabilities sum to one. The omitted rows are distinct token outputs, not one extra token. Greedy decoding chooses the maximum; sampling can choose another token. Softmax computes probabilities and has no learned parameters; selection is a separate operation. During training, cross-entropy against the known next token updates the vocabulary layer and Transformer. Later generation steps reuse the same projection.

What changes after we predict the next token? Trace each word to its input embedding eᵢ and final contextualised embedding eᵢ. Within a block, attention retrieves mᵢ; the output projection gives Δeᵢ; add it to eᵢ, then apply the MLP and its residual. The amber output is the last supplied position, which feeds the vocabulary head. Input embeddings include positions. The input and updated-embedding labels distinguish the stages without introducing a superscript. The displayed continuation is an example sequence, not a model run. Under causal attention, appending a word does not change earlier positions’ final embeddings for a fixed model in evaluation. The new position reads the available prefix; caching can reuse earlier computations. Multiple head messages are concatenated before the output projection. Normalisation is omitted from the diagram.

What changes after we predict the next token? Trace each word to its input embedding eᵢ and final contextualised embedding eᵢ. Within a block, attention retrieves mᵢ; the output projection gives Δeᵢ; add it to eᵢ, then apply the MLP and its residual. The amber output is the last supplied position, which feeds the vocabulary head. Input embeddings include positions. The input and updated-embedding labels distinguish the stages without introducing a superscript. The displayed continuation is an example sequence, not a model run. Under causal attention, appending a word does not change earlier positions’ final embeddings for a fixed model in evaluation. The new position reads the available prefix; caching can reuse earlier computations. Multiple head messages are concatenated before the output projection. Normalisation is omitted from the diagram.

What makes this a decoder-only model, and what is self-attention attending to? Follow the three supplied tokens into the repeated blocks and their three updated embeddings. Only the last embedding feeds the displayed next-token head. Read the causal square in token order: Raghav, goes, to. Append school and repeat. Colored cells mark allowed attention, not its magnitude; input embedding lookup and normalization are omitted in this summary. This is the standard text-only decoder-only architecture: no separate source encoder and no encoder-to-decoder cross-attention. Each block has causal multi-head self-attention and an MLP, with residual connections; normalization is omitted here. L counts repeated blocks. Training normally predicts the next token at every eligible position using a shifted target, while generation reads the last position. The same vocabulary head is shared across positions. The matrix shows allowed access, not measured attention. Decoding includes a probability distribution and token-selection step, expanded on the preceding slides.

Does a prompt and an answer imply an encoder-decoder architecture? Read each joined strip as one sequence. Purple tokens are supplied; amber tokens are appended one at a time. Only tokens generated so far are inputs at the next step. Follow the shared computation below and the return arrow after token selection. The two rows are separate illustrative completions, not measured model outputs or two inputs processed together. In every self-attention layer, Q, K and V come from the same current sequence. Read the final embedding to produce vocabulary logits, apply softmax and select a token. Decoder-only models can also translate, summarize and chat when suitably trained: the instruction and any source text remain inside the prompt. Those task names do not determine the architecture. A standard Transformer encoder-decoder instead has a separate source encoder whose outputs are available to the decoder through cross-attention. Discuss that distinction after the translation example. Capability depends on training and data; the architecture alone does not guarantee useful completions.

Do we need to generate a new sentence? Point to Raghav and Delhi in the same source sentence used on the opening slide.

What should the unhighlighted words receive? Count all six outputs, not just the two entity names. The four-class inventory is PERSON, ORGANIZATION, PLACE and OTHER. This sentence does not have an organization, but that class remains available at every position.

Why is it legitimate for Raghav’s state to use Delhi? Compare the full matrix with the familiar causal triangle. Do not introduce a new attention equation.

Do we need a new attention operation to read the whole input? Trace the Raghav query and the source-wide keys and values. Within one head/layer, positions share the projection matrices. We use row-vector notation: q₁ = e₁ × W_Q, K = E × W_K, V = E × W_V. e₁ is Raghav’s current embedding; E stacks all current input embeddings for this layer. The retrieved result m₁ is its attention message. Q, K and V in the projection subscripts identify the learned maps. Scaling uses the query/key width dₖ, not necessarily the model width.

Is e₁ just the original embedding for Raghav? Distinguish the input token identity from its final vector. The drawn bars indicate multiple features, not semantic coordinates.

Are we training six unrelated classifiers? Trace both Raghav and Delhi through the same wide classifier band. Ask which parameters are shared.

Does the classifier for Delhi still score PERSON? Point at all four scores, then say that the same computation repeats for every input position. A typical supervised loss sums or averages token cross-entropies. The task head must be trained; merely attaching a new head does not teach labels.

Must an encoder reduce the input to a single vector? Trace each of the six input positions to its updated output embedding. The full attention square means every supplied position can read every other position. The six separate output arrows retain one embedding per input position; this does not pool the sentence into a single vector. BERT is an example of this encoder family. Its pretraining objectives are not needed to understand the arrangement here.

We have five contextual states. How do we get one class prediction? Pause with all five states visible before suggesting a pooling operation.

What shape does mean pooling produce? Follow the five state arrows into the mean and then into the classifier. Remember this pooling drawing for translation. Mean pooling is a possible design, not a guarantee of useful features. It must be used with suitable training. This example has no padding.

How can a token at the beginning collect information from later words? Trace the added input position through the same stack. Follow only its final state to the sentiment head. CLS also has keys and values and uses the ordinary per-head/layer projection matrices. The encoder now returns six states for five review words plus CLS; the classifier reads one.

Do we initialize CLS with a new random vector for every sentence? Point to the ordinary token rows, then the CLS row. Follow the lookup arrow into the encoder input. The initial row is shared across sentences at any fixed set of model parameters; the updated CLS embedding depends on the supplied sentence. The table is a model parameter, not a fresh random draw for each example. Backpropagation and optimizer steps learn its CLS row. This is the standard special-token lookup design for text; a separate trainable vector is another implementation choice. The coloured cells are schematic vector coordinates, not measured values.

Do we provide a target embedding for CLS, or a true label y? Follow the solid forward arrows left to right. For the positive review, the illustrative probabilities are NEG 0.2 and POS 0.8; cross-entropy is −log(0.8), about 0.223. Then follow the dashed lower lane from right to left. Finally distinguish gradient computation from the optimizer step. Here ŷ denotes the full predicted probability vector, while y is the supplied class label, not another prediction. The loss uses the probability assigned to the true class; no argmax is inserted in the differentiable training path. The classifier drawing is a schematic MLP with one hidden layer; a linear classifier is also possible. Neuron counts do not specify the encoder width. Input embeddings receive positional information; all encoder positions participate even though only the CLS output is drawn. The dashed lane is an offset view of gradients through the same computational graph, not a direct shortcut from loss to CLS. This shows full fine-tuning of unfrozen parameters. The optimizer changes model parameters, not the stored value of a per-example updated CLS activation. Frozen parameters would not update. Natural logarithms are used, and the probabilities are illustrative.

What changes when the whole sentence needs one label? Compare the identical Raghav sentence on both sides. The sentence-classification label is deliberately unspecified until a dataset defines the task. For example, one could define a topic classification dataset; this drawing does not invent a measured topic prediction.

What does a single row of this matrix represent? Read the Raghav row across all eight cells, then count all six rows. The cells represent vector coordinates, not actual values. This is one sentence with no batch dimension. The width eight is chosen only to make the shape easy to draw. It does not come from the six-word sentence or the four entity classes.

What happens before we multiply by the classifier weights? Follow Raghav straight down, then Delhi. At every stage there are six token positions. The encoder changes their features while preserving width eight. The task head changes the feature width from eight to four class scores; its parameters are shared across all six positions. We retain eᵢ at position i and label its input and updated stages explicitly. The model width eight is illustrative. W has shape 8 × 4; b has four components and is added, not multiplied. In matrix form, scores = E W + b, with b broadcast across the six rows. The four output slots per word use PERSON, ORGANIZATION, PLACE, OTHER order; they are not numerical predictions. The next slide rearranges those six sets of scores into rows and applies softmax independently within each row. Positional information and normalization are omitted from the drawing.

Why is the result 6 × 4 rather than one vector of four scores? The previous slide showed four score slots beneath every word. Here each word is a row and each class is a column. Point across the Raghav row, then the Delhi row. Both use the same classifier parameters. ORG abbreviates ORGANIZATION. Dark cells indicate the illustrative highest-scoring class; their shading is not a numerical score or probability. Raghav is labelled PERSON, Delhi PLACE, and the other words OTHER in this teaching example. The table depicts score slots and chosen labels, not a trained model run.

Does selecting CLS shrink every embedding to one feature? Hold the eight columns fixed while highlighting the CLS row. The other rows still exist; this task head simply does not read them directly. CLS can contain information from the other words because its embedding has already passed through whole-input attention. This is row selection, not averaging the rows or compressing the feature dimension.

Does the classifier receive a one-hot CLS vector? Trace the top strip from token identity through lookup and encoding, then follow the arrow to the eight input neurons. Count six hidden units and three output neurons. Read each logit and its probability beside EDUCATION, TRAVEL and OTHER. The softmax operates across all three logits. The one-hot vector has |V| entries, one for each vocabulary token; it is not a three-class target vector. Implementations normally index the embedding table directly rather than materialize one-hot vectors. Positions are included before the encoder. The schematic classifier is an MLP: h = activation(e_CLS W₁ + b₁), with W₁ of shape 8 × 6 and b₁ of length 6; logits = h W₂ + b₂, with W₂ of shape 6 × 3 and b₂ of length 3. This is a valid alternative to a linear 8 × 3 head. The logit example [2, 1, 0] gives softmax probabilities approximately [0.665, 0.245, 0.090], summing to one before rounding. These are authored numbers, not model measurements. A labelled dataset and an annotation rule define the topic classes. The model width, hidden width, vocabulary size and class count are independent choices.

Does readout mean another token or another operation after CLS? Read each concrete choice aloud. The top row selects updated CLS for a sentence classifier. The middle row sends all token embeddings through a shared token classifier. The bottom row forms a whole-input representation by pooling or by selecting updated CLS. The diagram abbreviates the task heads in the first two rows. A sentence or token classifier is still needed to produce labels. Retrieval can use a suitably trained representation, potentially with a learned projection and normalization; neither raw pooling nor raw CLS is automatically useful for similarity. The training objective and evaluation must match the use.

Do contextual source vectors alone produce the Hindi sentence? Notice Delhi moves earlier in the Hindi sentence. Avoid suggesting one fixed source position maps to each output position. The translation is authored. Displayed Hindi words are teaching units, not a claim about real model tokenization.

Is शुरू the first word of the translation? Point to the brackets: these mark special tokens. Trace the start ID to its embedding-table row, then read the complete example output. We do not supply the complete target sentence during generation. The marker labels are a teaching convention, not a tokenizer configuration change. A real model uses its configured decoder start ID, which can be a BOS, EOS, pad or other designated ID. Here we use a simple BOS/EOS teaching design. The start row is a learned parameter loaded from a checkpoint or initialized before training. Each displayed Hindi word is treated as one token for clarity; actual tokenizers may split it. The target embedding is updated through the decoder while its table row is fixed during inference.

What new operation have we introduced so far? Keep the source strip, encoder band and vector positions fixed for all the following builds.

Where have we already used this operation? Reveal the pooling arrows below the unchanged encoder states. This first design gives the decoder one pooled source vector. We will choose how the decoder accesses the source in the following section.

What information does the decoder need at this step? Add the familiar causal decoder beneath c. Show the target prefix entering from the left. The diagram states a conditional dependency. It does not specify an extra token or a particular source interface. We first show the dependency, then compare concrete implementations.

What changes between these three predictions? Read the three rows in order. Keep pointing back to the same c. This frame describes p(yₜ | y<ₜ, c). The following frames implement that dependency in three different ways; the plus sign on this frame means both inputs are available, not necessarily numerical addition. Continue through स्कूल, जाता and है, then predict [समाप्त]. The outputs are teaching examples.

Can we add c directly to a target embedding? Trace the same source encoder and pool into c, then follow the projection and the repeated addition at the target positions. This is a concrete teaching architecture, not a required definition of an encoder-decoder. The decoder input has the same number of positions as the target prefix. The source encoder, projection and decoder are trained together on next-target-token loss. The source vector is recomputed for each source sentence and held fixed while that sentence is decoded. Normalisation and biases are omitted for clarity.

What does “add positions” actually add? Follow each vertical column. The source context u is shared; the token embedding and position vector differ. Point at each plus node and count d features before and after addition. This is an additive positional-encoding design. Position vectors may be learned or fixed sinusoidal vectors; their construction is not needed here. Other Transformer designs incorporate positions differently, for example through rotations of queries and keys. We index illustrated positions from one for teaching. Add the positional information once, after the displayed source-context fusion. For concatenation, add pᵢ after W_F has returned the fused vector to width d. For prepending, the source prefix occupies position 1, so the three target positions receive p₂, p₃ and p₄.

Is the decoder using self-attention again? Read each matrix row as one receiving target position. The gray में column is a teaching placeholder for a future token, not an actual input at this generation step. These embeddings already contain u and positional information. Their queries, keys and values are all target-side projections: this is self-attention. There is no separate query-to-English-token lookup in this fixed-context design. During parallel training, future supplied target positions are masked; during this generation step, they are absent. The same rule applies after concatenation and projection. With a prepended source vector, that extra earlier position also provides a key and value.

Where does the attention message go? Point from q₃ to the three keys, then from the weighted values to m₃. Trace the same message, delta and residual convention from the earlier attention lecture. The displayed equations describe one head. Multiple heads compute their own messages; concatenate them before W_O. Layer normalization is omitted from the drawing. Here eᵢ denotes the current embedding entering the attention sublayer: in the first block this is the prepared eᵢ + u + pᵢ from the previous slide. Scores use q₃ kᵢ transpose divided by sqrt(d_k). Repeat the block updates before reading the last position through the vocabulary head. The third embedding belongs to दिल्ली, not to the token being predicted.

What is the “last embedding,” and why read that one? Trace all three prepared inputs to all three updated outputs. Highlight the last one, then follow it through logits, softmax and token selection. Append में, obtain its token embedding and position vector, and run the next step. The subscript of e₃ identifies the third target token, not the token being predicted. The earlier updated embeddings still exist; generation uses the last target position to choose the next token. |V| is the target vocabulary size. Softmax does not itself choose the token. In the addition design, every new target input also receives the same source vector u for this source sentence. Concatenation and prepending change input preparation; the causal blocks and last-target-position readout follow the same pattern. Normalization and the individual attention-head projections are omitted from this summary.

Does concatenating c create another token? Point to the two feature groups, then the projection back to d. Count the target positions: there are still three. Apply the same fusion map at every target position. Train it with the source encoder and decoder through the translation loss. With a linear fusion, [eₜ ∥ c]W_F can be split into a learned transform of eₜ plus a learned transform of c; these are related conditioning designs, not claims of strictly different expressive power. This diagram is an authored teaching construction.

Why place u before शुरू rather than after the target prefix? Count four input positions. Add p₁ to u, p₂ to the शुरू embedding, p₃ to राघव and p₄ to दिल्ली. Keep e₁, e₂ and e₃ as target-token indices. Read updated e₃, at the fourth total input position, through the vocabulary head and softmax to predict में. The continuous prefix is a vector, not a vocabulary word to predict. Assign positions consistently, including the extra slot, and omit a next-token loss on the source slot in this teaching construction. Train the projection, source encoder and decoder on target next-token loss. Unlike the later cross-attention architecture, this uses one pooled source vector and causal self-attention. The appendix enlarges the same prefix construction.

What are the two sequences in this architecture? Name the family now, after following the computation through both stacks.

So what is the problem with using one vector? Trace encoder outputs through pooling into c. Show the separate growing target prefix. Then ask what source information is needed for who, where and when. There is no direct edge from the source embeddings to the decoder. Each displayed word is one token for this teaching example, so the source has eight positions. The Hindi items are translated fragments illustrating different information needs, not consecutive next-token predictions. Pooling produces a fixed-width vector even when the source grows longer. This creates a representation and learning burden; it does not prove that a fixed vector must forget information. The decoder can use c differently as its target prefix changes. It may guess a missing detail from learned patterns, but cannot retrieve it from the source. The next architecture keeps all source embeddings available for a fresh lookup.

What could we retain instead of just c? Return to the identical source drawing. Remove the pooling constraint while leaving the encoder unchanged.

When predicting राघव, which source position might receive more attention? Keep the encoder drawing fixed from the previous slide. Read all six weights, then point to शुरू as the supplied target prefix and राघव as the upcoming prediction. The weights sum to one. Real attention heads can use broader or different patterns; a name need not receive the largest weight in every head or layer. Crucially, the query for predicting the first Hindi token comes from the शुरू position. Feeding राघव to predict itself would leak the target. The following slides explain the source of the query, keys, values and weights.

What information is available from the source? Stop at the blue encoder outputs. Do not introduce target queries or source keys and values yet. This sequence expands one head in one decoder block. English source indices and Hindi target indices are separate. Source e₁ belongs to Raghav; the target e₁ introduced next belongs to [शुरू].

Where does the first target embedding come from? Keep the encoder outputs fixed. Reveal only the start token, embedding lookup and position addition. [शुरू] is the teaching label for the special beginning-of-sequence token. It is not an ordinary word in the translation. During inference, lookup reads the learned table row; it does not train that row.

With only [शुरू] supplied, what can target self-attention read? Trace input e₁ plus position through self-attention and the residual to updated e₁. There is no token prediction at this stage. With a single unmasked key, the attention weight is one, but the value and output projections can still produce a nonzero update. Once more Hindi tokens are supplied, causal self-attention can use their preceding target context. Layer normalization is omitted. The target MLP follows cross-attention in this decoder block.

Is this the query used in the preceding target self-attention? Reveal the new arrow from updated e₁ to q₁. Keep the e₁ subscript because this is still the start position. The cross-attention query will be matched against English source keys. W_Q here means the cross-attention query projection; the self-attention sublayer has a different W_Q. The local symbol is reused for the same mathematical role, not to imply shared weights. In implementations with pre-normalization, the projection takes a normalized version of this updated target stream. Normalization is omitted from the drawing.

How are the English keys and values constructed? Reveal the six key/value pairs and the query-to-key connections. Follow the English paths down into K and V, then the updated target path into Q. Cross-attention has its own learned W_Q, W_K and W_V, separate from target and source self-attention parameters. Source keys and values can be computed in advance; this reveal order is for teaching and does not impose an execution order. Keys determine source-position weights; values supply the message. The query at [शुरू] supports prediction of राघव after the remaining decoder computation.

How do the illustrative weights become information the decoder can use? Trace source scores downward through softmax to the weighted values. The larger 0.72 weights the Raghav value more heavily in this illustration. Keep source-attention weights distinct from output-vocabulary probabilities. To make the arithmetic internally consistent, the authored scaled scores are log(0.72), log(0.04), log(0.03), log(0.09), log(0.04), log(0.08); displayed scores are rounded to two decimals. Softmax of the unrounded scores reproduces the weights exactly. These are not trained model scores. c₁ is a cross-attention message, analogous to mᵢ in self-attention. For multiple heads, concatenate their messages before W_O. The decoder retains the target stream through the residual; c₁ alone is neither the final embedding nor a vocabulary distribution.

What exactly changes when शुरू attends to the English sentence? Follow the शुरू stream, then pause at the fork: one copy makes the query and the other follows the long residual arrow. Trace the source message through W_O into the plus node. The result is still the embedding at शुरू. This is one head in one decoder block; with multiple heads, concatenate their source messages before W_O. Layer normalization is omitted. The residual adds to the current target representation entering cross-attention, which has already passed through target self-attention; it does not jump back to the raw embedding-table row. The message c₁ plays the role of the earlier attention message mᵢ. The shown 0.72 weight is illustrative, not measured attention. A forward pass updates this contextual representation; it does not by itself train or overwrite the embedding table. Here the source is read through K/V, not through an added fixed pooled u.

Does the updated शुरू embedding become the embedding of राघव? Trace the updated शुरू embedding through the MLP, final vocabulary scores, probabilities and chosen token. Point to both positions in the new prefix: शुरू remains first, राघव is second. The decoder repeats causal self-attention, cross-attention, and MLP sublayers with their residuals across its blocks. The top row shows the continuation of the block enlarged on the preceding slide. Read the final target embedding after all blocks. Each new token receives its input embedding and positional information. English encoder outputs, and each layer’s source keys and values, can be reused while the target prefix grows. Predictions are illustrative.

Why do we need both keys and values? Follow the K path into the scores, then the separate V path into the weighted sum. The query affects the sum through its weights. E contains final encoder embeddings. eₜ is the current target embedding after causal self-attention and before the cross-attention update. cₜ is the retrieved message, playing the same role as mᵢ earlier. Project the message to an embedding update, add it to the target stream, then apply the MLP and its residual. Different layers and heads generally use different projection parameters.

Which dimension does this softmax normalize? Connect each equation to its corresponding arrow in the preceding diagram. Separate scores, weights and the weighted message. One head is shown. Multiple head messages are concatenated and projected back to model width before combining with the target stream. All equations use row-vector notation.

What changes when we use several cross-attention heads? Trace the same E and D into all three heads, then join their outputs. Students already know the multi-head calculation; stop after identifying the two origins. For head j: Qⱼ = D W_Q⁽ʲ⁾, Kⱼ = E W_K⁽ʲ⁾, Vⱼ = E W_V⁽ʲ⁾. Cⱼ = softmax(QⱼKⱼᵀ / √dₖ)Vⱼ. W_O maps the concatenated messages to model width before the residual update.