Lecture 1 - Basic Principles

September 1st, 2026

Finish basic ReLU net.
Differences and parallels between ML and DL
Introduce ERM, talk about expectations. Challenges in using training/test data paradigm.
Introduce the idea of regularization for overfitting. Challenge of too many hyperparameters.
Gradient descent, convergence (and stopping) and connection to eigenvalues
Ridge regression/Kernel form.