Stanford CME295 Transformers & LLMs | Autumn 2025 | Lecture 1 - Transformer

About this video

• Video Title: Stanford CME295 Transformers & LLMs | Autumn 2025 | Lecture 1 - Transformer
• Channel: Stanford Online
• Speakers: Afshin Amidi, Shervine Amidi
• Duration: 01:41:59

Overview

This lecture provides an introduction to Transformers and Large Language Models (LLMs), covering foundational concepts in Natural Language Processing (NLP). It begins with an overview of NLP tasks, tokenization, and word representation, then delves into Recurrent Neural Networks (RNNs) and the attention mechanism. The lecture culminates in an explanation of the Transformer architecture and a detailed example to illustrate these concepts.

Key takeaways

• NLP Overview: Natural Language Processing (NLP) tasks are categorized into classification, multi-classification, and generation, with examples like sentiment analysis, named entity recognition, and machine translation.
• Tokenization and Representation: Text is broken down into tokens, and various methods like word-level, subword, and character-level tokenization are discussed, along with the need for meaningful embeddings to represent these tokens.
• RNNs and Limitations: Recurrent Neural Networks (RNNs) process text sequentially, maintaining a hidden state, but struggle with long-range dependencies and vanishing gradients.
• Attention Mechanism: The attention mechanism allows models to directly link different parts of the input text, improving the handling of long-range dependencies and enabling contextual understanding.
• Transformer Architecture: The Transformer model utilizes self-attention and cross-attention mechanisms, along with positional encodings, to process input and generate output sequences, forming the basis for modern LLMs.