Motion graphic

Generative AI Language Modeling with Transformers

9 hours · 2 modules · PyTorch + Hugging Face

The most important technical course in the certificate. Everything after this is applying transformers. If you only truly understand one course, make it this one.

---

Module 1 — Fundamental Concepts of Transformer Architecture

Why transformers exist

RNNs process sequentially — token n cannot be computed until token n−1 is done. Two consequences: no parallelism (so training is slow and doesn't exploit GPUs), and information from distant tokens degrades over many steps.

Attention already existed as a patch on seq2seq. The 2017 paper's move was to remove the RNN and keep only attention. Result: full parallelism across sequence positions, and a constant-length path between any two tokens.

Positional encoding

Attention is permutation-invariant — shuffle the input and the output set is unchanged. So position must be injected explicitly.

PE(pos, 2i)   = sin(pos / 10000^(2i/d_model))
PE(pos, 2i+1) = cos(pos / 10000^(2i/d_model))

Added (not concatenated) to the token embeddings.

Why sinusoids:

• Each dimension is a sinusoid of a different wavelength — high-frequency dims encode fine

position, low-frequency dims encode coarse position. A unique fingerprint per position.

• Bounded in [−1, 1], so it doesn't swamp the embedding
• Relative position is expressible as a linear function of PE, so the model can learn

"attend 3 tokens back"

• Extrapolates beyond training lengths

Alternatives you'll meet in practice: learned positional embeddings (BERT, GPT-2 — simpler, doesn't extrapolate) and RoPE rotary embeddings (Llama and most current models — rotates Q and K by a position-dependent angle, giving better long-context behaviour).

class PositionalEncoding(nn.Module):
    def __init__(self, d_model, max_len=5000):
        super().__init__()
        pe  = torch.zeros(max_len, d_model)
        pos = torch.arange(0, max_len).unsqueeze(1).float()
        div = torch.exp(torch.arange(0, d_model, 2).float() *
                        (-math.log(10000.0) / d_model))
        pe[:, 0::2] = torch.sin(pos * div)
        pe[:, 1::2] = torch.cos(pos * div)
        self.register_buffer('pe', pe.unsqueeze(0))    # buffer: saved, but not a parameter

    def forward(self, x):
        return x + self.pe[:, :x.size(1)]

register_buffer — part of the model state, moves with .to(device), but not trained.

Attention

The core operation. Every token asks a question, every token advertises what it offers, and the answer is a weighted blend.

Q, K, V — three linear projections of the input:

• Query — what this token is looking for
• Key — what each token offers
• Value — what each token actually contributes

Database analogy: you have a query, you match it against keys, you retrieve a blend of values weighted by match strength. Unlike a database it's soft — you get a bit of everything, proportional to relevance.

Attention(Q,K,V) = softmax( Q·Kᵀ / √d_k ) · V

Step by step:

1. Q·Kᵀ — every query against every key → an n×n score matrix
2. / √d_k — scaling. Without it, dot products grow with dimension, softmax saturates into

near-one-hot, and gradients vanish. This is why it's called scaled dot-product attention, and it's a favourite exam question.

1. softmax — scores become weights summing to 1 per row
2. · V — weighted sum of values

Self-attention means Q, K and V all come from the same sequence — every token attends to every token including itself. Cross-attention takes Q from the decoder and K, V from the encoder; that's how a decoder consults the input in a translation model.

Multi-head attention

Run h attention operations in parallel with different learned projections, concatenate, project back.

head_i    = Attention(Q·W_i^Q, K·W_i^K, V·W_i^V)
MultiHead = Concat(head_1 … head_h) · W^O

Why: one attention pattern can only express one kind of relationship. Different heads specialise — syntactic dependencies, coreference, positional patterns. It's an ensemble of relationship-detectors inside one layer.

Each head works in d_model / h dimensions, so total compute matches single-head attention at full width. With d_model=512 and h=8, each head is 64-dimensional.

attn = nn.MultiheadAttention(embed_dim=512, num_heads=8, batch_first=True)
out, weights = attn(query, key, value, key_padding_mask=mask)

batch_first=True — otherwise PyTorch expects (seq, batch, feature), which is a classic silent-shape-bug source.

Masking

Two distinct kinds, and mixing them up is a common error:

Padding mask — ignore [PAD] positions. Needed in every model that batches variable-length sequences.

Causal / look-ahead mask — position i may not attend to positions > i. Required for autoregressive generation, or the model trivially cheats by reading the answer.

mask = torch.triu(torch.ones(n, n), diagonal=1).bool()    # upper triangle = blocked
scores = scores.masked_fill(mask, float('-inf'))          # -inf → 0 after softmax

-inf rather than 0 because the mask is applied before softmax; e^(-inf) = 0, so the position gets exactly zero weight and the remaining weights still sum to 1.

The transformer block

x → MultiHeadAttention → Dropout → + x → LayerNorm      (Add & Norm)
  → FeedForward(Linear→ReLU/GELU→Linear) → Dropout → + → LayerNorm

Residual connections give gradients a direct path — the same ResNet idea from Course 5, and the reason you can stack 96 of these blocks.

LayerNorm, not BatchNorm — normalises per sample across features, so it's independent of batch size and of sequence length, and behaves identically in training and inference.

The feed-forward network is applied position-wise (the same MLP at every position), typically expanding 4× (512 → 2048 → 512). Most of a transformer's parameters live here, not in attention.

Pre-norm (x + Attn(LN(x))) vs post-norm (LN(x + Attn(x))): the original paper used post-norm; modern LLMs use pre-norm because it trains more stably at depth. Labs use either.

Encoder for classification

encoder_layer = nn.TransformerEncoderLayer(d_model=512, nhead=8,
                                           dim_feedforward=2048, dropout=0.1,
                                           batch_first=True)
encoder = nn.TransformerEncoder(encoder_layer, num_layers=6)

class Classifier(nn.Module):
    def __init__(self, vocab, d_model, nhead, nlayers, nclass):
        super().__init__()
        self.emb  = nn.Embedding(vocab, d_model)
        self.pos  = PositionalEncoding(d_model)
        self.enc  = nn.TransformerEncoder(
            nn.TransformerEncoderLayer(d_model, nhead, batch_first=True), nlayers)
        self.fc   = nn.Linear(d_model, nclass)

    def forward(self, x, mask=None):
        h = self.pos(self.emb(x) * math.sqrt(self.emb.embedding_dim))
        h = self.enc(h, src_key_padding_mask=mask)
        return self.fc(h.mean(dim=1))          # mean-pool, or take the [CLS] position

The * sqrt(d_model) scaling of embeddings before adding positional encoding is in the original paper — it keeps embedding magnitude comparable to the positional signal.

Efficient training

Named in the "Optimization Techniques" video, and all of it is standard practice:

• Mixed precision (fp16/bf16) — ~2× faster, half the memory
• Gradient accumulation — simulate a large batch on small hardware by accumulating gradients

over several steps before optimizer.step()

• Gradient checkpointing — recompute activations in the backward pass instead of storing

them; trades compute for memory

• Learning-rate warmup + decay — transformers are unstable at high LR early; warm up linearly

then decay (cosine or inverse-sqrt)

• AdamW — Adam with decoupled weight decay. The standard optimiser for transformers.
• Gradient clipping — clip_grad_norm_(params, 1.0) against exploding gradients
• Flash Attention — an I/O-aware exact attention kernel; same maths, far less memory traffic

---

Module 2 — Advanced Concepts of Transformer Architecture

Decoder models (GPT-like)

Decoder-only, trained on causal language modelling: predict the next token given all previous ones.

"The cat sat on the" → "mat"

Every position predicts its successor, so a sequence of n tokens gives n training signals in one forward pass. Self-supervised — the labels are the input, shifted by one. That's why the entire internet is training data.

The causal mask is what makes it work. Without it the model sees the answer.

class GPTLike(nn.Module):
    def __init__(self, vocab, d_model, nhead, nlayers):
        super().__init__()
        self.emb = nn.Embedding(vocab, d_model)
        self.pos = PositionalEncoding(d_model)
        layer = nn.TransformerDecoderLayer(d_model, nhead, batch_first=True)
        self.dec = nn.TransformerDecoder(layer, nlayers)
        self.fc  = nn.Linear(d_model, vocab)

    def forward(self, x):
        n = x.size(1)
        causal = nn.Transformer.generate_square_subsequent_mask(n).to(x.device)
        h = self.pos(self.emb(x))
        h = self.dec(h, memory=h, tgt_mask=causal)
        return self.fc(h)

Training targets are the inputs shifted left:

inputs  = tokens[:, :-1]
targets = tokens[:, 1:]
loss = F.cross_entropy(logits.reshape(-1, vocab), targets.reshape(-1))

Generation is a loop — predict, append, repeat:

for _ in range(max_new_tokens):
    logits = model(idx[:, -context:])[:, -1, :] / temperature
    probs  = F.softmax(logits, dim=-1)
    nxt    = torch.multinomial(probs, num_samples=1)
    idx    = torch.cat([idx, nxt], dim=1)

Note [:, -1, :] — only the final position's logits matter for the next token.

Hugging Face equivalent:

from transformers import AutoModelForCausalLM, AutoTokenizer
tok   = AutoTokenizer.from_pretrained('gpt2')
model = AutoModelForCausalLM.from_pretrained('gpt2')
out   = model.generate(**tok("Once upon a time", return_tensors='pt'),
                       max_new_tokens=50, do_sample=True, top_p=0.9, temperature=0.8)
print(tok.decode(out[0], skip_special_tokens=True))

Encoder models (BERT)

Encoder-only, bidirectional — every token sees the full sentence in both directions. That's why BERT is strong at understanding and cannot generate autoregressively.

Two pretraining objectives:

MLM (Masked Language Modelling) — randomly mask 15% of tokens and predict them. Of that 15%: 80% replaced with [MASK], 10% replaced with a random token, 10% left unchanged.

Why that 80/10/10 split — a genuine exam question: [MASK] never appears at fine-tuning time, so training exclusively on it creates a train/inference mismatch. The random and unchanged cases force the model to build a good representation of every token, not just masked positions.

NSP (Next Sentence Prediction) — given sentence A and sentence B, did B actually follow A? 50% real pairs, 50% random. Intended to teach inter-sentence relationships for QA and inference tasks. Later work (RoBERTa) showed NSP contributes little and dropped it — worth knowing, since the course teaches NSP and interviewers often mention its removal.

Input format:

[CLS] sentence A tokens [SEP] sentence B tokens [SEP]

plus segment/token-type embeddings marking which sentence each token belongs to. BERT therefore sums three embeddings: token + position + segment.

The [CLS] output vector is the sentence-level representation used for classification.

from transformers import BertForMaskedLM, BertTokenizer
tok   = BertTokenizer.from_pretrained('bert-base-uncased')
model = BertForMaskedLM.from_pretrained('bert-base-uncased')

inputs = tok("The capital of France is [MASK].", return_tensors='pt')
logits = model(**inputs).logits
idx    = (inputs.input_ids == tok.mask_token_id).nonzero()[0,1]
tok.decode(logits[0, idx].argmax())      # 'paris'

Data preparation for BERT pretraining — what the lab walks through:

1. Split the corpus into sentence pairs
2. Build NSP labels (50% consecutive, 50% random second sentence)
3. Tokenize with [CLS] / [SEP]
4. Apply the 80/10/10 masking to 15% of tokens
5. Build attention_mask and token_type_ids
6. Two losses: MLM cross-entropy over masked positions + NSP binary cross-entropy

Encoder–decoder for translation

The full original architecture.

source → Encoder (bidirectional self-attention) → memory
target → Decoder (causal self-attention, then CROSS-attention over memory) → output

Three attention mechanisms in the complete model, and naming them is a common exam item:

1. Encoder self-attention — source attends to source, unmasked
2. Decoder masked self-attention — target attends to earlier target only
3. Cross-attention — decoder queries, encoder keys and values. This is where the translation

actually happens.

model = nn.Transformer(d_model=512, nhead=8, num_encoder_layers=6,
                       num_decoder_layers=6, batch_first=True)
out = model(src, tgt,
            src_key_padding_mask=src_pad,
            tgt_mask=nn.Transformer.generate_square_subsequent_mask(tgt.size(1)),
            tgt_key_padding_mask=tgt_pad)

Training uses teacher forcing (feed the true target, shifted). Inference is autoregressive: start from [BOS], append each predicted token, stop at [EOS] or max length.

The three families, decided

Why nearly all modern LLMs are decoder-only: the next-token objective is uniformly applicable to all text, it scales cleanly, and it turns out that with enough scale a decoder can do understanding tasks too — by being prompted. Simpler architecture, one objective, better scaling.

---

Course 9 in one page

• Attention: softmax(QKᵀ/√d_k)V. Q = what I want, K = what I offer, V = what I give
• √d_k prevents softmax saturation and dead gradients
• Multi-head = parallel relationship-detectors in d_model/h dimensions each
• Positional encoding is mandatory — attention is permutation-invariant
• Padding mask hides [PAD]; causal mask hides the future; apply as -inf before softmax
• Block = Attention → Add&Norm → FFN → Add&Norm; residuals make depth trainable; LayerNorm not BatchNorm
• Most parameters live in the FFN, not attention
• GPT = decoder + causal LM + generation. BERT = encoder + MLM/NSP + understanding.

T5/BART = both + transformation

• BERT sums token + position + segment embeddings; [CLS] is the sentence vector
• Cross-attention (decoder Q, encoder K/V) is where translation happens
• Attention is O(n²) in sequence length — the origin of every context-window limit
• Train with AdamW + warmup + clipping + mixed precision