Information Theory, Coding and Inference (2026)
Table of Contents
Welcome to EE3800/EE5903 (Information Theory, Coding and Inference).
This is the official webpage of the EE3800/EE5903 course for 2026. All classes will be conducted in person. Online resources and references will be shared here. Much of our online interaction (homework submissions, announcements, off-classroom discussions, etc) will be on Google classroom. Invites will be sent to registered students by the first week of August. If you have not received an invite by the second class, please send me an email.
Information theory has its roots in this paper by Claude Shannon, and dealt with the following problems:
- Data compression
- Communication over noisy channels
Since its inception, information theory has proved to be a valuable tool in several fields including cryptography, physics, machine learning and computational biology. We will cover the basics of information theory, and discuss applications to communications, statistical inference and machine learning.
Prerequisites
- Strong foundation in probability and random processes
- Programming in python
- Some knowledge of digital communications will be helpful, but not mandatory
1. Assessment (tentative):
Each student will be expected to
- attend classes and participate actively
- solve exams and in-class quizzes
- solve programming assingments
| Exams | 70% | |
| Surprise quizzes | 10% | |
| Programming assignments | 20% |
If you are planning to audit the course, you must secure a regular pass grade to be eligible for an AU grade.
2. Instructor:
| Name | Dr. Shashank Vatedka |
| shashankvatedka@ee.iith.ac.in | |
| Office | EE616, EECS building |
3. Class timings:
- Slot R:
- Class venue: A221
4. Primary references:
We will primarily use the following material for the course:
- Elements of Information Theory by Thomas Cover and Joy Thomas
- Information Theory: From Coding to Learning by Yury Polyanskiy and Yihong Wu (draft copy from author website)
- Information Theory: Coding Theorems for Discrete Memoryless Systems by Imre Csiszar and Janos Korner
- Information theory, inference, and learning algorithms, David MacKay (soft copy available for free on the author’s website). A very nice book, but gives a very different flavour that what will be covered in the course. Suggested for additional reading.
Other references:
- “A mathematical theory of communication”, Claude Shannon, Bell systems Tech Journal, 1948. The classic paper that started this field.
- A student’s guide to coding and information theory, Stefan Moser (amazon link).
- Similar courses offered at IISc, Stanford, and MIT. Also one on Coursera.
Light reading:
- The information: A history, a theory, a flood James Gleick
- A mind at play: How Claude Shannon invented the information age Jimmy Soni and Rob Goodman. A biography of Claude Shannon.
To recap basics in probability and random processes:
- Probability, Random Variables and Stochastic Processes, Athanasios Papoulis and Unnikrishna Pillai
- Probability with Engineering Applications by Bruce Hajek
- Random Processes for Engineers by Bruce Hajek
5. Tentative list of topics
6. Class notes
7. Academic honesty and plagiarism
Students are encouraged to discuss with each other regarding class material and assignments. However, verbatim copying in any of the assignments or exams (from your friends or books or an online sources) is not allowed. This includes programming assignments. It is good to collaborate when solving assignments, but the solutions and programs must be written on your own. Copying in assignments or exams will result in a fail grade.
See this page (maintained by the CSE department), this page, and this one to understand more about plagiarism.