Schedule
Lectures are Tuesday and Thursday, 9:30–10:59 AM in The Gateway Building 1210. Discussion sections meet on Friday. Homework is nominally due on Wednesday at 10:59 PM. Class notes and iPad notes are linked from the schedule as they are posted.
The topic schedule is tentative and will be adjusted during the semester based on student and instructor feedback, as well as developments in the field.
Week 0: Introduction
| Tue, Aug 25 | ||
| Wed, Aug 26 | ||
| Thu, Aug 27 | Lecture 0Intros, Logistics, high-level themes/goals Class notes |
|
| Fri, Aug 28 |
Week 1: Basic Principles and Optimization
| Tue, Sep 01 | Lecture 1Basic Principles Class notes |
|
| Wed, Sep 02 | ||
| Thu, Sep 03 | ||
| Fri, Sep 04 |
Week 2: Optimization and History
| Tue, Sep 08 | ||
| Wed, Sep 09 | ||
| Thu, Sep 10 | Lecture 4History and Retrospective |
|
| Fri, Sep 11 |
Week 3: Misfitting
| Tue, Sep 15 | Lecture 5Misfitting |
|
| Wed, Sep 16 | ||
| Thu, Sep 17 | Lecture 6Guest lecture (Sizhe) Slides |
|
| Fri, Sep 18 |
Week 4: Maximal Update Parameterization (μP)
| Tue, Sep 22 | Lecture 7RMS RMS norm, Maximal update parameterization (μP) iPad notes |
|
| Wed, Sep 23 | ||
| Thu, Sep 24 | Lecture 8MuP iPad notes |
|
| Fri, Sep 25 |
Week 5: Convolutional Neural Networks
| Tue, Sep 29 | Lecture 9Convolutional neural nets iPad notes |
|
| Wed, Sep 30 | ||
| Thu, Oct 01 | Lecture 10Convolutional neural nets (Continued) |
|
| Fri, Oct 02 |
Week 6: Convolutional and Graph Neural Networks
| Tue, Oct 06 | Lecture 11Convolutional neural nets (Continued) iPad notes |
|
| Wed, Oct 07 | ||
| Thu, Oct 08 | Lecture 12Graph Neural Nets iPad notes |
|
| Fri, Oct 09 |
Week 7: Graph Neural Networks, RNNs, and Self-Supervision
| Tue, Oct 13 | Lecture 13Finish GNN: Pooling, Start RNN iPad notes |
|
| Wed, Oct 14 | ||
| Thu, Oct 15 | Lecture 14RNNs, Self-supervision |
|
| Fri, Oct 16 |
Week 8: Self-Supervision and State-Space Models
| Tue, Oct 20 | Lecture 15Self-supervision, State-space models iPad notes |
|
| Wed, Oct 21 | ||
| Thu, Oct 22 | Lecture 16State-space models iPad notes |
|
| Fri, Oct 23 |
Week 9: State-Space Models and Attention
| Tue, Oct 27 | Lecture 17State space models Continued iPad notes |
|
| Wed, Oct 28 | ||
| Thu, Oct 29 | Lecture 18Attention iPad notes |
|
| Fri, Oct 30 |
Week 10: Transformers
| Tue, Nov 03 | Lecture 19Attention recap, Transformer architecture, Embedding layer, Positional encoding iPad notes |
|
| Wed, Nov 04 | ||
| Thu, Nov 05 | Lecture 20Guest lecture (Mert) iPad notes |
|
| Fri, Nov 06 |
Week 11: Transformer Architectures, Embeddings, Soft Prompting, LoRA
| Tue, Nov 10 | Position Encoding, GPT-3, BERT, Other architectures |
|
| Wed, Nov 11 | Veterans Day — campus closed
|
|
| Thu, Nov 12 | Lecture 21Embeddings, soft prompting, LoRA iPad notes |
|
| Fri, Nov 13 |
Week 12
| Tue, Nov 17 | Lecture 22 |
|
| Wed, Nov 18 | ||
| Thu, Nov 19 | Lecture 23 |
|
| Fri, Nov 20 | Discussion 13 |
Week 13
| Tue, Nov 24 | Lecture 24 |
|
| Wed, Nov 25 | Homework 12 Due Worksheet |
|
| Thu, Nov 26 | Holiday (Thanksgiving) |
|
| Fri, Nov 27 | Thanksgiving holiday — campus closed, no discussion sections
|
Week 14
| Tue, Dec 01 | Lecture 25 |
|
| Wed, Dec 02 | Homework 13 Due |
|
| Thu, Dec 03 | Lecture 26 |
|
| Fri, Dec 04 | Discussion 14 |