When | What |
---|---|
Week 1 | |
Wed Aug 28 |
Introduction and Course Overview
[PDF] [PPTX]
Topics: Turing Test, Overview of LLMs (How do LLMs work, What LLMs can do, Limitations of LLMs, What is the future), Course Logistics. HW0 released. |
Week 2 | |
Mon Sep 02 | No class - Labor Day |
Wed Sep 04 |
Background
[PDF] [PPTX]
Topics: Language Modeling; Perplexity Evaluation; Feedforward Networks. |
Sun Sep 08 | HW0 due. |
Week 3 | |
Mon Sep 09 |
From Pytorch to Hugging Face: How to run your own LLM
[PDF] [PPTX] [notebook]
Topics: Pytorch framework; Hugging Face Transformers library. HW1 released. |
Wed Sep 11 |
The Pre-Transformer Era
[PDF] [PPTX]
Topics: RNNs; their Variants, Applications, and Limitations; Seq2Seq architecture; Attention mechanism. |
Week 4 | |
Mon Sep 16 |
The Transformer Architecture: Part I
[PDF] [PPTX]
Topics: Impact of Transformers; From Recurrence to Attention; Transformer block. |
Wed Sep 18 |
The Transformer Architecture: Part II
[PDF] [PPTX]
Topics: Overall Transformer Model; Inference and Training; SOTA Transformer Case Studies; Drawbacks of Transformers. |
Wed Sep 18 | HW1 Part 1 due. |
Week 5 | |
Mon Sep 23 |
Guest Lecture by Yann Dubois, Ph.D. Candidate, Stanford University
Topics: LLM Benchmarking and Evaluation. |
Wed Sep 25 |
Guest Lecture by Hanjun Dai, Staff Research Scientist & Research Manager, Google Brain
Topics: RLHF and LLM Reward Modeling. |
Wed Sep 25 | HW1 Part 2 due. |
Week 6 | |
Mon Sep 30 |
Adaptation - Part 1
Topics: TBD. HW2 released. |
Wed Oct 02 | Adaptation - Part 2 |
Wed Oct 02 | HW1 due. |
Week 7 | |
Mon Oct 07 |
Fast and Efficient Inference
Topics: TBD. |
Wed Oct 09 |
Guest Lecture by Denny Zhou, Principal Scientist & Research Director, Google DeepMind
Topics: LLM Reasoning. |
Week 8 | |
Mon Oct 14 |
Guest Lecture by Hyung Won Chung, Research Scientist, OpenAI
Topics: TBD. HW3 released. |
Wed Oct 16 |
Guest Lecture by Guangxuan Xiao, Ph.D. Candidate, MIT
Topics: Quantization. |
Sun Oct 20 | HW2 due. |
Week 9 | |
Mon Oct 21 |
Guest Lecture by Kai Sheng Tai, Research Scientist, Meta AI
Topics: Efficient Inference. |
Wed Oct 23 |
RAG and Vector DBs
Topics: TBD. |
Week 10 | |
Mon Oct 28 |
Guest Lecture by Ram Sriharsha, Chief Technology Officer, Pinecone
Topics: LLM Hallucinations, RAG, Vector DBs. |
Wed Oct 30 |
Agent Frameworks
Topics: TBD. |
Week 11 | |
Mon Nov 04 | Guest Lecture by TBD |
Wed Nov 06 |
Neurosymbolic Architectures
Topics: TBD. HW4 released. |
Sun Nov 10 | HW3 due. |
Week 12 | |
Mon Nov 11 |
Neurosymbolic Architectures
Topics: TBD. |
Wed Nov 13 | Guest Lecture by TBD |
Week 13 | |
Mon Nov 18 |
Project Introduction Presentations on December 2, 4, and 9 |
Wed Nov 20 | Guest Lecture by Jason Wei, Member of the Technical Staff, OpenAI |
Week 14 | |
Mon Nov 25 | Guest Lecture by Aakanksha Chowdhery, Senior Staff Research Scientist, Meta |
Wed Nov 27 | No class - Work on HW4 and Project |
Sun Dec 01 | HW4 due. |
Week 15 | |
Mon Dec 02 | Project Presentation Day 1 |
Wed Dec 04 | Project Presentation Day 2 |
Week 16 | |
Mon Dec 09 | Project Presentation Day 3 |