Deep Learning (2026)
Table of Contents
EE6380
I am teaching EE6380 Deep Learning in Jul-Nov 2026.
The goal of this course is to provide a rigorous introduction to the techniques and principles of deep learning. A lot of deep learning involves clever heuristics, and there are a number of counterintuitive phenomena. However, many of the basic principles are based on rigorous learning-theoretic foundations. We will aim to highlight these basic principles and connect them to practice. Using this as a foundation, we will try to develop intuition for some of the heuristic techniques that are popularly used. At the same time, we will also see some empirically successful practical deep learning techniques that traditional learning theory cannot explain.
This is the official course webpage. Links for lecture notes and references will be posted here.
Much of our online interaction (homework submissions, announcements, off-classroom discussions, etc) will be on Moodle (Updated link!). You can log in using your IITH email id, and is accessible only within campus (or VPN). This requires a pass key to join, which will be sent to registered students by the end of the first class. If you have not received an invite by the second class, please send me an email.
Prerequisites
- Strong foundation in probability/random processes, and matrix theory.
- Pattern Recognition and Machine Learning/Foundations of Machine Learning
- Programming in python (Enough comfort to do linear regression or classification on a dataset from scratch)
- A background in convex optimization and/or statistical learning theory is a plus, but not necessary
1. Assessment (tentative):
Each student will be expected to
- attend classes and participate actively
- solve exams and in-class quizzes
- solve programming assingments
| Exams (3) | 60% | |
| Programming assignments | 15% | |
| Programming challenges + viva | 20% | |
| Attendance | 5% |
Tentative dates for exams:
- Aug 29
- Oct 11
- Nov 15
If you are planning to audit the course, you must secure a regular pass grade to be eligible for an AU grade.
2. Instructor:
| Name | Dr. Shashank Vatedka |
| shashankvatedka@ee.iith.ac.in | |
| Office | EE616, EECS building |
Timings:
- Slot S
- Class venue: LHC-11
3. References:
We will not follow any single reference for the course. I will post rough lecture notes/slides (see the section below).
Some useful references:
- Deep Learning: Foundations and Concepts, Christopher Bishop and Hugh Bishop
- Deep Learning, Ian Goodfellow, Yoshua Bengio and Aaron Courville
- Dive Into Deep Learning, Aston Zhang, Zachary Lipton, Mu Li and Alexander Smola
For basics of machine learning and statistical learning theory:
- Understanding Machine Learning: From Theory to Algorithms by Shai-Shalev Schwartz and Shai-Ben David (free pdf from author website)
- Foundations of Machine Learning by Mehryar Mohri, Afshin Rostamizadeh and Ameet Talwalkar
- Learning Theory from First Principles by Francis Bach (pdf)
To recap basics in probability and random processes:
- Probability, Random Variables and Stochastic Processes, Athanasios Papoulis and Unnikrishna Pillai
- Probability with Engineering Applications by Bruce Hajek
- Random Processes for Engineers by Bruce Hajek
To recap basics in linear algebra/matrix theory:
- Linear algebra and its applications, Gilbert Strang
- Linear algebra done right, Sheldon Axler
For the basics of optimization:
- Convex Optimization, Stephen Boyd and Lieven Vandenburghe
- Algorithms for Convex Optimization, Nisheeth Vishnoi
- Convex Optimization: Algorithms and Complexity, Sebastian Bubeck
4. Tentative list of topics
- Statistical learning theory foundations
- Stochastic Gradient Descent, Backpropagation
- Classification and logistic regression
- Basics of deep learning on pytorch
- Modern optimization algorithms for deep learning
- Convolutional Neural Networks
- Sequence modeling and Recurrent Neural Networks, LSTM and GRU
- Transformers
- State space models, basics of language models
- Generative modeling
5. Class notes/recordings
Class notes/slides will be uploaded regularly to this Google Drive folder.
Recorded lectures will be posted to this YouTube playlist.
6. Programming challenge 1
6.1. Logistics
The challenge is to design a convolutional neural network for a wireless communication problem. You do not require any background on digital/wireless communications to solve this, and you may treat this as any other image classification probelm. You are given only the training and validation datasets. Separate test datasets will be used to evaluate your models.
The challenge proceeds in two stages:
- Oct 27: Submit your model, this will be evaluated on the test dataset and your final accuracy will appear on a public leaderboard. This model will NOT be used for grading, and is to only give you an idea of where you may approximately stand in your leaderboard. You can make further modifications to your model after this.
Nov 1: Submit your final model, along with your report, and the code used to train the final model. The final model will be evaluated on the test dataset, and used for grading. We may run your (training) code to make sure that it works, and gives the same model/performance (use fixed seeds). You must ensure that your model can be trained on Google Colab in a reasonable timeframe.
You are allowed the use of generative AI to help you with these challenges, but you must write the report in your own words, and must be able to explain every step of the process and every line of code. If you use generative AI, you must explicitly declare which model you used, how you used it, and your entire conversation/process as a pdf document separate from your final report. You are permitted to use AI to generate this second document. Your final report must contain a section where you briefly mention what part of the solution was obtained using AI, and what ideas you brought into the solution.
Your performance on the final leaderboard will determine 40% of your score. The remaining 60% (based on the report and viva) will test your effort and understanding. In particular, topping the leaderboard without any understanding of the process involved may not give you a high score.
6.2. The problem and dataset
This is a classification problem. Your goal is to learn a detector for a communication system. There are 8 types of input signals (8 classes), each of which is passed through an unknown communication medium, and modified by practical input, output and channel impairments. You have access to the spectrogram (computed from the magnitude of the short-term Fourier transform) of the received signal in each case. The dataset therefore consists of images (corresponding to the spectrogram) and the true labels. You must design a suitable convolutional neural network for this problem.
See this page for details.
7. Academic honesty and plagiarism
Students are encouraged to discuss with each other regarding class material and assignments. However, verbatim copying in any of the assignments or exams (from your friends or books or an online sources) is not allowed. This includes programming assignments. It is good to collaborate when solving assignments, but the solutions and programs must be written on your own. Copying in assignments or exams will result in a fail grade.
The usage of AI to understand concepts is encouraged, but you are not permitted to use this for homework submissions.
See this page (maintained by the CSE department), this page, and this one to understand more about plagiarism.