Course 7 of 13 · Generative AI / LLMs

Generative AI and LLMs: Architecture and Data Preparation

The pivot point — from training models to adapting foundation models. Tokenization, data quality, and the GPT/BERT/T5 map

6 hours · 2 modules · PyTorch, NLTK, spaCy, Hugging Face tokenizers

The pivot point of the certificate. Courses 1–6 were about training models. From here on you are an AI Engineer: you take enormous pretrained models and adapt them. This course sets the vocabulary for everything that follows.

Module 1 — Generative AI Architecture

Discriminative vs generative

• Discriminative models learn P(y|x) — a decision boundary. "Is this spam?"
• Generative models learn P(x) or P(x,y) — the data distribution itself. They can sample

new data from it.

That's the whole distinction, and it's the first quiz question.

The architectures, and what each is for

RNN / LSTM — sequential, hidden state carries memory. Superseded for language, still used for small time-series work. Weakness: no parallelism, weak long-range memory.

Transformer — self-attention, fully parallel, long-range. The architecture behind every current LLM.

VAE (Variational Autoencoder) — encoder produces a distribution (μ, σ) over a latent space; sample it and decode to generate. Loss = reconstruction + KL divergence. Smooth, interpolatable latent space; outputs tend to be blurry.

GAN — generator vs discriminator, adversarial. Sharp outputs, one-shot generation, unstable training, mode collapse.

Diffusion — learn to reverse a gradual noising process. Currently dominant for images (Stable Diffusion, DALL·E, Midjourney). Stable training, excellent quality, slow generation because it's iterative.

The LLM families — the table to memorise

The mental model:

• Encoder-only sees the whole sequence bidirectionally → best for understanding
• Decoder-only sees only leftward context → best for generating
• Encoder–decoder does both → best for transforming one sequence into another

Ask "does this task need to read, write, or translate?" and the family follows.

Generative AI for NLP

Tasks the videos enumerate: text generation, summarisation, translation, question answering, sentiment analysis, code generation, chatbots/dialogue, named entity recognition.

The shift the course is arguing for: previously each task needed a task-specific model trained on task-specific labelled data. Now one foundation model handles all of them via prompting, with fine-tuning only where you need domain adaptation.

Hallucination

An LLM produces fluent, confident, false output. Not a bug you can patch — a consequence of the objective. The model is trained to predict plausible next tokens, and plausible is not the same as true. It has no internal fact-checking mechanism and no representation of its own uncertainty.

Causes: gaps in training data, outdated training data, ambiguous prompts, over-confident decoding, and the model's tendency to complete a pattern rather than decline.

Mitigations, in rough order of effectiveness:

1. RAG — ground answers in retrieved documents (Courses 12–13; this is the main answer)
2. Fine-tuning on domain data (Courses 10–11)
3. Prompt engineering — ask for citations, permit "I don't know"
4. Lower temperature for factual tasks
5. Human review for anything consequential

Why it matters here: this is why the last third of the certificate is about RAG. The architecture answer to hallucination is don't ask the model to recall — give it the document.

watsonx.governance

IBM's product for AI governance: model lifecycle tracking, bias/drift monitoring, explainability, and regulatory compliance documentation. It appears because it's IBM's course. Know the concept — AI governance = monitoring, documentation, risk and compliance for deployed models — and that it's a real interview topic under the EU AI Act.

Libraries you'll use

• Hugging Face Transformers — pretrained models, tokenizers, pipeline(), Trainer
• PyTorch — the substrate
• NLTK / spaCy — classical NLP, tokenisation, POS tagging
• LangChain — LLM application framework (Courses 12–13)
• Datasets — HF data loading
• PEFT / TRL — parameter-efficient fine-tuning and RLHF (Courses 10–11)

The fastest possible LLM call, which the lab has you run:

from transformers import pipeline

gen = pipeline('text-generation', model='gpt2')
print(gen("The future of AI is", max_length=50, num_return_sequences=1))

clf = pipeline('sentiment-analysis')
print(clf("This course is dense but useful."))

pipeline() handles tokenisation, model loading, inference and decoding in one call. Good for demos; you'll drop to the lower-level API when you need control.

Module 2 — Data Preparation for LLMs

Tokenization

Splitting text into the units a model actually processes. Everything downstream depends on it.

Word-based — split on whitespace/punctuation.

• Intuitive; vocabulary explodes; every unseen word is <UNK>; "run/running/ran" are unrelated

entries.

Character-based — tiny vocabulary (~100), no OOV ever.

• Sequences become very long, and each token carries almost no meaning.

Subword — the compromise every modern LLM uses. Frequent words stay whole; rare words break into meaningful pieces. unhappiness → un + happi + ness. Fixed vocabulary, no OOV, morphology partly captured.

Subword algorithms:

• BPE (Byte-Pair Encoding) — start from characters, repeatedly merge the most frequent

adjacent pair. Used by GPT-2, RoBERTa.

• WordPiece — like BPE but merges by likelihood gain rather than raw frequency. Used by

BERT. Marks continuations with ##.

• SentencePiece / Unigram — language-agnostic, treats input as a raw byte/character stream

so it needs no pre-tokenisation and handles languages without spaces. Used by T5, XLNet, ALBERT.

import nltk
nltk.word_tokenize("Don't split this wrongly.")
# ['Do', "n't", 'split', 'this', 'wrongly', '.']

import spacy
nlp = spacy.load('en_core_web_sm')
[t.text for t in nlp("Apple is looking at buying a U.K. startup.")]

from transformers import BertTokenizer, XLNetTokenizer
bt = BertTokenizer.from_pretrained('bert-base-uncased')
bt.tokenize("Tokenization is unbelievably important")
# ['token', '##ization', 'is', 'unbelievable', '##ly', 'important']   ← ## = continuation

xt = XLNetTokenizer.from_pretrained('xlnet-base-cased')
xt.tokenize("Tokenization is important")
# ['▁Token', 'ization', '▁is', '▁important']                          ← ▁ = word start

Two conventions to keep straight: BERT/WordPiece marks continuation with ##; XLNet and SentencePiece mark word start with ▁.

Special tokens

From tokens to tensors

enc = tokenizer("Hello world", padding=True, truncation=True,
                max_length=128, return_tensors='pt')
enc['input_ids']       # token IDs
enc['attention_mask']  # 1 = real token, 0 = padding

attention_mask is not optional. It tells the model to ignore padding positions. Without it, padding tokens participate in attention and corrupt the representation — silently.

Three steps, always in this order: tokenize → numericalize → pad.

Data loaders for NLP

Text samples have different lengths, so batching needs a collate_fn:

from torch.nn.utils.rnn import pad_sequence
from torch.utils.data import DataLoader

def collate_fn(batch):
    texts, labels = zip(*batch)
    ids = [torch.tensor(tokenizer.encode(t)) for t in texts]
    ids = pad_sequence(ids, batch_first=True, padding_value=tokenizer.pad_token_id)
    return ids, torch.tensor(labels)

loader = DataLoader(dataset, batch_size=32, shuffle=True, collate_fn=collate_fn)

collate_fn runs after __getitem__ gathers a batch and before it's handed to your loop — it's where padding belongs. Pad per batch, not to a global maximum: padding everything to the longest document in the dataset wastes enormous compute.

For torchtext-style pipelines the course also shows build_vocab_from_iterator, vocab objects, and <unk> default index handling.

Data quality and diversity

The section with the highest real-world value in this course.

Quality dimensions:

• Accuracy — factually correct
• Diversity — domains, registers, demographics, languages
• Relevance — matched to intended use
• Cleanliness — deduplicated, boilerplate stripped, encoding fixed
• Ethics — licensed, consented, PII removed

Why it dominates model quality: at fixed compute, data quality beats data quantity. Deduplication alone measurably improves models — duplicated passages get memorised and crowd out learning. Recent small-model results (training on curated/textbook-quality corpora) beat much larger models trained on raw scrapes.

Bias in → bias out. A model trained on internet text reproduces the internet's demographic skews, stereotypes and coverage gaps. There is no debiasing step downstream that fully repairs a skewed corpus — which is exactly why data curation is a job, and why it connects directly to the Human Data Lead work.

The preprocessing pipeline:

1. Collect
2. Deduplicate (exact and near-duplicate; MinHash/LSH at scale)
3. Filter for quality (heuristics, classifiers, perplexity filters)
4. Remove PII and toxic content
5. Normalise (encoding, whitespace, unicode)
6. Tokenize
7. Shuffle and shard

Course 7 in one page

• Discriminative learns boundaries P(y|x); generative learns the distribution P(x)
• GPT = decoder = generate. BERT = encoder = understand. T5/BART = both = transform.
• Diffusion beat GANs on images: stable training, no mode collapse, at the cost of slow sampling
• Hallucination is inherent to next-token prediction; RAG is the architectural answer
• Subword tokenization (BPE / WordPiece / SentencePiece) — fixed vocab, no OOV
• ## = WordPiece continuation, ▁ = SentencePiece word start
• Special tokens: [CLS] [SEP] [PAD] [UNK] [MASK]
• tokenize → numericalize → pad; pad per batch in collate_fn; always send attention_mask
• Match the tokenizer to the model — mismatches fail silently
• Data quality and deduplication beat data volume; bias in the corpus is bias in the model