Kernel Methods (2024)
Table of Contents
Welcome to EE5605 (Kernel Methods).
This is a 1-credit course and all classes will be conducted in person. Online resources and references will be shared here. Much of our online interaction (homework submissions, announcements, off-classroom discussions, etc) will be on Google classroom. Invites will be sent to registered students by the first week of the 5-6 segment. If you have not received an invite by the second class, please send me an email.
This course will delve into the problem of binary classification, and fundamental techniques to solve this efficiently. We will first look at linear classifiers, and see how the problem of finding a good classifier can be posed as an optimization problem that can be solved efficiently. We will then see the limitations of linear classifiers and then look at kernel methods, which are a powerful tool in supervised learning problems.
The course will be mathematical in nature, and some of the assignment questions will involve programming.
Prerequisites
- Strong foundation in probability and random processes
- EE5603/AI2200 Concentration Inequalities
- EE5604 Intro to SLT
- Comfort with programming in python. We will mainly use the numpy library.
1. Assessment (tentative):
Each student will be expected to
- attend classes and participate actively
- solve homework assignments
- solve 2 exams
| Mid-term exam | 40% | Week of 8th Nov (tentative) |
| Final exam | 60% | Week of 25th Nov |
If you are planning to formally audit the course, you must secure a regular pass grade to be eligible for an AU grade.
2. Instructor:
| Name | Dr. Shashank Vatedka |
| shashankvatedka@ee.iith.ac.in | |
| Office | EE616, New EECS building |
3. Class timings:
- Slot P: Mondays 14:30-16:00, Thursdays 16:00-17:30
- Class venue: A-221
4. Primary references:
We will primarily use the following material for the course:
- Understanding Machine Learning: From Theory to Algorithms by Shai-Shalev Schwartz and Shai-Ben David (free pdf from author website)
- Foundations of Machine Learning by Mehryar Mohri, Afshin Rostamizadeh and Ameet Talwalkar
- Learning Theory from First Principles by Francis Bach (pdf)
- Lecture notes on SLT by Percy Liang (link)
References for concentration inequalities (which we will use to some extent in the course)
- Concentration Inequalities by Stephane Boucheron, Gabor Lugosi and Pascal Massart
- High-dimensional Probability by Roman Vershynin
- “Introduction to statistical learning theory” by Olivier Bousquet, Stephane Boucheron and Gabor Lugosi
To recap basics in probability and random processes:
- Probability, Random Variables and Stochastic Processes, Athanasios Papoulis and Unnikrishna Pillai
- Probability with Engineering Applications by Bruce Hajek
- Random Processes for Engineers by Bruce Hajek
5. Tentative list of topics
- Linear classification using Support Vector Machines
- Hard and soft SVM, sample complexity
- Stochastic Gradient Descent
- Kernel methods
6. Class notes and recordings
Class notes will be uploaded regularly to this folder.
Some class recordings will be uploaded to this YouTube playlist.
7. Academic honesty and plagiarism
Students are encouraged to discuss with each other regarding class material and assignments. However, verbatim copying in any of the assignments or exams (from your friends or books or an online sources) is not allowed. This includes programming assignments. It is good to collaborate when solving assignments, but the solutions and programs must be written on your own. Copying in assignments or exams will result in a fail grade.
See this page (maintained by the CSE department), this page, and this one to understand more about plagiarism.