Deep Learning (2026)
Table of Contents
EE6380
I am teaching EE6380 Deep Learning in Jul-Nov 2026.
The goal of this course is to provide a rigorous introduction to the techniques and principles of deep learning. A lot of deep learning involves clever heuristics, and there are a number of counterintuitive phenomena. However, many of the basic principles are based on rigorous learning-theoretic foundations. We will aim to highlight these basic principles and connect them to practice. Using this as a foundation, we will try to develop intuition for some of the heuristic techniques that are popularly used. At the same time, we will also see some empirically successful practical deep learning techniques that traditional learning theory cannot explain.
This is the official course webpage. Links for lecture notes and references will be posted here.
Much of our online interaction (homework submissions, announcements, off-classroom discussions, etc) will be on Moodle (link). You can log in using your IITH email id, and is accessible only within campus (or VPN). This requires a pass key to join, which will be sent to registered students by the end of the first class. If you have not received an invite by the second class, please send me an email.
Prerequisites
- Strong foundation in probability/random processes, and matrix theory.
- Pattern Recognition and Machine Learning/Foundations of Machine Learning
- Programming in python (Enough comfort to do linear regression or classification on a dataset from scratch)
- A background in convex optimization and/or statistical learning theory is a plus, but not necessary
1. Assessment (tentative):
Each student will be expected to
- attend classes and participate actively
- solve exams and in-class quizzes
- solve programming assingments
| Exams | 70% | |
| Surprise quizzes | 10% | |
| Programming assignments | 20% |
If you are planning to audit the course, you must secure a regular pass grade to be eligible for an AU grade.
2. Instructor:
| Name | Dr. Shashank Vatedka |
| shashankvatedka@ee.iith.ac.in | |
| Office | EE616, EECS building |
timings:
- Slot S
- Class venue: LHC-11
3. References:
We will not follow any single reference for the course. I will post rough lecture notes/slides (see the section below).
Some useful references:
- Deep Learning: Foundations and Concepts, Christopher Bishop and Hugh Bishop
- Deep Learning, Ian Goodfellow, Yoshua Bengio and Aaron Courville
- Dive Into Deep Learning, Aston Zhang, Zachary Lipton, Mu Li and Alexander Smola
For basics of machine learning and statistical learning theory:
- Understanding Machine Learning: From Theory to Algorithms by Shai-Shalev Schwartz and Shai-Ben David (free pdf from author website)
- Foundations of Machine Learning by Mehryar Mohri, Afshin Rostamizadeh and Ameet Talwalkar
- Learning Theory from First Principles by Francis Bach (pdf)
To recap basics in probability and random processes:
- Probability, Random Variables and Stochastic Processes, Athanasios Papoulis and Unnikrishna Pillai
- Probability with Engineering Applications by Bruce Hajek
- Random Processes for Engineers by Bruce Hajek
To recap basics in linear algebra/matrix theory:
- Linear algebra and its applications, Gilbert Strang
- Linear algebra done right, Sheldon Axler
For the basics of optimization:
- Convex Optimization, Stephen Boyd and Lieven Vandenburghe
- Algorithms for Convex Optimization, Nisheeth Vishnoi
- Convex Optimization: Algorithms and Complexity, Sebastian Bubeck
4. Tentative list of topics
- Statistical learning theory foundations
- Stochastic Gradient Descent, Backpropagation
- Classification and logistic regression
- Basics of deep learning on pytorch
- Modern optimization algorithms for deep learning
- Convolutional Neural Networks
- Sequence modeling and Recurrent Neural Networks, LSTM and GRU
- Transformers
- State space models, basics of language models
- Generative modeling
5. Class notes/recordings
Class notes/slides will be uploaded regularly to this Google Drive folder.
Recorded lectures will be posted to this YouTube playlist.
6. Academic honesty and plagiarism
Students are encouraged to discuss with each other regarding class material and assignments. However, verbatim copying in any of the assignments or exams (from your friends or books or an online sources) is not allowed. This includes programming assignments. It is good to collaborate when solving assignments, but the solutions and programs must be written on your own. Copying in assignments or exams will result in a fail grade.
The usage of AI to understand concepts is encouraged, but you are not permitted to use this for homework submissions.
See this page (maintained by the CSE department), this page, and this one to understand more about plagiarism.