EE6317 Course Information
Table of Contents
Welcome to EE6317 (Channel coding). This is a follow-up course to EE 2340 (Information Sciences) and EE 5847 (Information theory). This course will cover the basic concepts of coding for reliable communication over noisy channels. While not a strict prerequisite, it would be helpful if you have already taken EE5390 (Source coding).
Most of our interaction will take place on Google classroom. Please send me an email if you have not received an invite.
Classes will be online. We will try a mix of recorded videos (uploaded regularly on youtube, links shared on Google classroom), online discussion sessions (on Google meet), and live-streamed lectures.
1 Assessment (tentative):
Each student will be expected to
- watch lecture videos (2-3 hour videos per week)
- attend discussion sessions (1-2 times per week, in regular class hours)
- solve homework assignments, roughly 2-3 (collaboration is fine as long as it is explicitly declared, but you have to write the solutions in your own words), and participate in class/discussion sessions
- solve an online midterm exam (tentative) and a final exam
| Homeworks | 40% |
| Mid-term exam | 20% |
| Final exam | 40% |
2 Instructor:
| Name | Dr. Shashank Vatedka |
| shashankvatedka@ee.iith.ac.in |
3 Class timings:
- Slot F: Tuesdays 11:00-12:00, Wednesdays 14:30-15:30 and Fridays 10:00-11:00
- Class venue: Online
4 Textbook and references:
Primarily, the material will be drawn from
- Lecture notes and slides, uploaded here and Google classroom.
- Elements of Information Theory, (EIT) Thomas M Cover and Joy A Thomas, second edition, Wiley inc. (Amazon link). Limited number of copies available in the university library. While the first edition of the book has all the material we need for the course, the second edition has a much expanded problem set.
Other references:
- "A mathematical theory of communication", Claude Shannon, Bell systems Tech Journal, 1948. The classic paper that started this field.
- Modern Coding Theory, Tom Richardson and Ruediger Urbanke, Cambridge University Press, 2008. The bible for LDPC codes.
- Information theory, inference, and learning algorithms, David MacKay (soft copy available for free on the author's website). A very nice book, but gives a very different flavour that what will be covered in the course. Suggested for additional reading.
- A student's guide to coding and information theory, Stefan Moser (amazon link).
- Information theory: coding theorems for discrete memoryless systems, Imre Csiszar and Janos Korner. An advanced textbook. Recommended reading after covering this course as well as EE6317.
Light reading:
- The information: A history, a theory, a flood James Gleick
- A mind at play: How Claude Shannon invented the information age Jimmy Soni and Rob Goodman. A biography of Claude Shannon.
5 Tentative syllabus:
Recap of basic concepts in information theory; Channels and error correcting codes; Discrete memoryless channels; Shannon's channel coding theorem; Typical sets and the asymptotic equipartition property; Joint and conditionally typical sets; Proof of the channel coding theorem; Linear codes; Linear codes achieve capacity; Low-density parity-check (LDPC) codes; Belief propagation decoding of LDPC codes; Polar codes; Polar codes achieve capacity of binary-input memoryless symmetric channels;
6 Tentative schedule
| Topic | Date | Notes/links/material | Homeworks |
|---|---|---|---|
| Reliable communication over noisy channels | Week 1 | Review Chapter 2 EIT | |
| - Review of basic concepts | Review (video,notes) | ||
| - Discrete memoryless channels | Introduction (video,notes) | ||
| - Rate, encoding, decoding, complexity | Channel coding and capacity (video1,video2,notes) | ||
| - The channel coding theorem | Capacity of BEC (notes) | ||
| Reading: EIT Sec 7.1-7.5 | |||
| Proof of the channel coding theorem | Week 1-2 | Converse proof (video,notes) | |
| - Typical sets, AEP, concentration | Reading: EIT Sec 7.9 | ||
| - Random codes and proof of achievability | Typical sets (video1,video2,notes) | ||
| - Fano's inequality and proof of converse | Reading: EIT Sec 7.6 | ||
| - Gaussian channels | |||
| Linear codes | Week 3 | Introduction (video) | |
| - Construction | Linear codes achieve capacity of BSC (video) | ||
| - Encoding and decoding | References: 1 2 | ||
| - Linear codes achieve capacity | |||
| Low-density parity check codes | Week 3 | Introduction (video) | |
| - Introduction and construction | Decoding LDPC codes (video) | ||
| - Belief propagation decoding | Reference: Chapter 3 of Richardson and Urbanke | ||
| - Other aspects | |||
| Polar codes | Week 4 | Polar codes (video) | |
| - Introduction and construction | |||
| - Successive interference cancellation decoding | |||
| - Polar codes achieve capacity | |||
| - Discussion |