| When | What |
|---|---|
| Week 1 | |
| Wed Aug 28 |
Introduction and Course Overview
[PDF] [PPTX]
Topics: Turing Test, Overview of LLMs (How do LLMs work, What LLMs can do, Limitations of LLMs, What is the future), Course Logistics. HW0 released. |
| Week 2 | |
| Mon Sep 02 | No class - Labor Day |
| Wed Sep 04 |
Background
[PDF] [PPTX]
Topics: Language Modeling; Perplexity Evaluation; Feedforward Networks. |
| Sun Sep 08 | HW0 due. |
| Week 3 | |
| Mon Sep 09 |
From Pytorch to Hugging Face: How to run your own LLM
[PDF] [PPTX] [notebook]
Topics: Pytorch framework; Hugging Face Transformers library. HW1 released. |
| Wed Sep 11 |
The Pre-Transformer Era
[PDF] [PPTX]
Topics: RNNs; their Variants, Applications, and Limitations; Seq2Seq architecture; Attention mechanism. |
| Week 4 | |
| Mon Sep 16 |
The Transformer Architecture: Part I
[PDF] [PPTX]
Topics: Impact of Transformers; From Recurrence to Attention; Transformer block. |
| Wed Sep 18 |
The Transformer Architecture: Part II
[PDF] [PPTX]
Topics: Overall Transformer Model; Inference and Training; SOTA Transformer Case Studies; Drawbacks of Transformers. |
| Wed Sep 18 | HW1 Part 1 due. |
| Week 5 | |
| Mon Sep 23 |
Guest Lecture by Yann Dubois, Ph.D. Candidate, Stanford University
Topics: LLM Benchmarking and Evaluation. |
| Wed Sep 25 |
Guest Lecture by Hanjun Dai, Staff Research Scientist & Research Manager, Google Brain
Topics: RLHF and LLM Reward Modeling. |
| Wed Sep 25 | HW1 Part 2 due. |
| Week 6 | |
| Mon Sep 30 |
Adaptation - Part 1
Topics: TBD. HW2 released. |
| Wed Oct 02 | Adaptation - Part 2 |
| Wed Oct 02 | HW1 due. |
| Week 7 | |
| Mon Oct 07 |
Fast and Efficient Inference
Topics: TBD. |
| Wed Oct 09 |
Guest Lecture by Denny Zhou, Principal Scientist & Research Director, Google DeepMind
Topics: LLM Reasoning. |
| Week 8 | |
| Mon Oct 14 |
Guest Lecture by Hyung Won Chung, Research Scientist, OpenAI
Topics: TBD. HW3 released. |
| Wed Oct 16 |
Guest Lecture by Guangxuan Xiao, Ph.D. Candidate, MIT
Topics: Quantization. |
| Sun Oct 20 | HW2 due. |
| Week 9 | |
| Mon Oct 21 |
Guest Lecture by Kai Sheng Tai, Research Scientist, Meta AI
Topics: Efficient Inference. |
| Wed Oct 23 |
RAG and Vector DBs
Topics: TBD. |
| Week 10 | |
| Mon Oct 28 |
Guest Lecture by Ram Sriharsha, Chief Technology Officer, Pinecone
Topics: LLM Hallucinations, RAG, Vector DBs. |
| Wed Oct 30 |
Agent Frameworks
Topics: TBD. |
| Week 11 | |
| Mon Nov 04 | Guest Lecture by TBD |
| Wed Nov 06 |
Neurosymbolic Architectures
Topics: TBD. HW4 released. |
| Sun Nov 10 | HW3 due. |
| Week 12 | |
| Mon Nov 11 |
Neurosymbolic Architectures
Topics: TBD. |
| Wed Nov 13 | Guest Lecture by TBD |
| Week 13 | |
| Mon Nov 18 |
Project Introduction Presentations on December 2, 4, and 9 |
| Wed Nov 20 | Guest Lecture by Jason Wei, Member of the Technical Staff, OpenAI |
| Week 14 | |
| Mon Nov 25 | Guest Lecture by Aakanksha Chowdhery, Senior Staff Research Scientist, Meta |
| Wed Nov 27 | No class - Work on HW4 and Project |
| Sun Dec 01 | HW4 due. |
| Week 15 | |
| Mon Dec 02 | Project Presentation Day 1 |
| Wed Dec 04 | Project Presentation Day 2 |
| Week 16 | |
| Mon Dec 09 | Project Presentation Day 3 |