This Course ran in the Autumn term of 2018/2019
Class Times: Fridays 14:00 - 15:30 (Sometimes Wednesdays 11:30 - 13:00 see syllabus).
Location: IALS Lecture Theater LG26 Lecture Room, Bentham House, WC1H 0EG, Ground Floor Lecture Theater, Wilkins Building
Instructor: Carlo Ciliberto
TAs: Giulia Luise
Office Hours: By appointment. 3rd floor Hub, 66 Gower street
Email Contact : cciliber (a) gmail.com, g.luise.16 (a) ucl.ac.uk
This course represents half of Advanced Topics in Machine Learning (aka COMP GI13 / COMP M050) from the UCL CS MSc on Machine Learning.
The other half is Reproducing kernel Hilbert spaces in Machine Learning (Taught by Prof. Arthur Gretton).
Course announcements will be posted on the mailing list.
Statistical Learning Theory (SLT) studies the problem of learning from empirical observations (data) to predict and/or understand the behavior of an unknown phenomenon (e.g. the dynamics of the stock market or the activations patterns in the human brain). SLT provides a mathematical framework within which it is possible rigorously address questions such as "How to design a learning algorithm", "what does it mean for an algorithm to 'solve' a learning problem" or "How to compare two learning algorithms".
The goal of this course is to introduce students to the ideas behind most well-established learning algorithms and provide fundamental insights on how to use them in practice or to design new ones.
Linear Algebra, Probability Theory, Calculus.
Final grades will depend on two project assignment (50%) and a final exam (50%).
| Class | Date | Topic |
|---|---|---|
| 1 | Fri Oct 05 | Course Overview |
| 2 | Fri Oct 12 | Overfitting and Regularization I |
| 3 | Fri Oct 19 | Overfitting and Regularization II |
| 4 | Wed Oct 24 | Tikhonov Regularization |
| 5 | Fri Nov 02 | Generalization Error and Stability |
| 6 | Fri Nov 09 | Least-Squares Regression |
| 7 | Fri Nov 16 | Computational Regularization via Early Stopping I |
| 8 | Fri Nov 23 | Computational Regularization via Early Stopping II |
| 9 | Wed Nov 28 | Structured Prediction I |
| 10 | Fri Dec 14 | Structured Prediction II |
There is no required text for the course. Below is a number of useful references:
- T. Poggio and L. Rosasco course slides and videos.
- S. Shalev-Shwartz and S. Ben-David Understanding Machine Learning: From Theory to Algorithms (Online Book). Cambridge University Press , 2014.
- O. Bousquet, S. Boucheron and G. Lugosi Introduction to Statistical Learning Theory (Tutorial).
- P. Liang course notes.
- N. Cristianini and J. Shawe-Taylor. Kernel Methods for Pattern Analysis . Cambridge University Press, 2004.
- I. Steinwart and A. Christmann. Support Vector Machines Springer, 2008.