| When | What |
|---|---|
| Week 1 | |
| Wed Aug 28 |
Introduction and Course Overview
[PDF] [PPTX]
Topics: Turing test; overview of LLMs (how do LLMs work, what LLMs can do, limitations of LLMs, what is the future); course logistics. HW0 released. |
| Week 2 | |
| Mon Sep 02 | No class - Labor Day |
| Wed Sep 04 |
Background
[PDF] [PPTX]
Topics: language modeling; perplexity evaluation; feedforward neural networks. |
| Sun Sep 08 | HW0 due. |
| Week 3 | |
| Mon Sep 09 |
From Pytorch to Hugging Face: How to run your own LLM
[PDF] [PPTX] [notebook]
Topics: Pytorch framework; Hugging Face Transformers library. HW1 released. |
| Wed Sep 11 |
The Pre-Transformer Era
[PDF] [PPTX]
Topics: recurrent neural networks; RNN variants, applications, and limitations; Seq2Seq architecture; attention mechanism. |
| Week 4 | |
| Mon Sep 16 |
The Transformer Architecture: Part I
[PDF] [PPTX]
Topics: impact of transformers; from recurrence to attention; the transformer block. |
| Wed Sep 18 |
The Transformer Architecture: Part II
[PDF] [PPTX]
Topics: overall transformer architecture; inference and training; SOTA transformer case studies; drawbacks of transformers. |
| Wed Sep 18 | HW1 Part 1 due. |
| Week 5 | |
| Mon Sep 23 |
Guest Lecture by Yann Dubois, Ph.D. Candidate, Stanford University [slides] [video]
Title: Scalable Evaluation of Large Language Models |
| Wed Sep 25 |
Guest Lecture by Hanjun Dai, Staff Research Scientist & Research Manager, Google Brain [slides] [video]
Title: Preference Optimization for Large Language Models |
| Sun Sep 29 | HW1 Part 2 due. |
| Week 6 | |
| Mon Sep 30 |
Adaptation - Part 1
PPTX
Topics: TBD. HW2 released. |
| Wed Oct 02 | Adaptation - Part 2 PPTX |
| Week 7 | |
| Mon Oct 07 |
Fast and Efficient Inference
Topics: TBD. |
| Wed Oct 09 |
Guest Lecture by Denny Zhou, Principal Scientist & Research Director, Google DeepMind
Topics: LLM Reasoning. |
| Wed Oct 09 | HW1 Part 3 due. |
| Week 8 | |
| Mon Oct 14 | Guest Lecture by Hyung Won Chung, Research Scientist, OpenAI |
| Wed Oct 16 |
Guest Lecture by Guangxuan Xiao, Ph.D. Candidate, MIT
Topics: Quantization. |
| Week 9 | |
| Mon Oct 21 |
Guest Lecture by Kai Sheng Tai, Research Scientist, Meta AI
Topics: Efficient Inference. |
| Mon Oct 21 | HW2 Part 1 due. |
| Wed Oct 23 |
RAG and Vector DBs
Topics: TBD. |
| Week 10 | |
| Mon Oct 28 |
Guest Lecture by Ram Sriharsha, Chief Technology Officer, Pinecone
Topics: LLM Hallucinations, RAG, Vector DBs. |
| Mon Oct 28 | HW2 Part 2 due. |
| Wed Oct 30 |
Agent Frameworks
Topics: TBD. |
| Week 11 | |
| Mon Nov 04 | Guest Lecture by Thang Luong, Senior Staff Research Scientist, Google DeepMind |
| Wed Nov 06 |
Neurosymbolic Architectures
Topics: TBD. |
| Fri Nov 08 |
HW2 Part 3 due. HW3 released. |
| Week 12 | |
| Mon Nov 11 |
Neurosymbolic Architectures
Topics: TBD. |
| Wed Nov 13 | Guest Lecture by TBD |
| Fri Nov 15 | HW3 Part 1 due. |
| Week 13 | |
| Mon Nov 18 |
Project Work Presentations on December 2, 4, and 9 |
| Wed Nov 20 | Guest Lecture by Jason Wei, Member of the Technical Staff, OpenAI |
| Fri Nov 22 | HW 3 Part 2 due. |
| Week 14 | |
| Mon Nov 25 | Guest Lecture by Aakanksha Chowdhery, Senior Staff Research Scientist, Meta |
| Wed Nov 27 | No class - Work on Project |
| Week 15 | |
| Mon Dec 02 | Project Presentation Day 1 |
| Wed Dec 04 | Project Presentation Day 2 |
| Week 16 | |
| Mon Dec 09 | Project Presentation Day 3 |