X
X
X

X
Courses » An Introduction to Information Theory

An Introduction to Information Theory

ABOUT THE COURSE

Information Theory answers two fundamental questions: what is the maximum data rate at which we can transmit over a communication link, and what is the fundamental limit of data compression. In this course we will explore answers to these two questions. We will study some practice source compression algorithms. We will also study how to compute channel capacity of simple channels.

INTENDED AUDIENCE

3rd/4th year UG students in EC stream, 1st year PG students in communications and signal processing specialization

PRE-REQUISITES

Basic knowledge of probability theory and digital communications

INDUSTRY SUPPORT – LIST OF COMPANIES/INDUSTRY THAT WILL RECOGNIZE/VALUE THIS ONLINE COURSE

Communication companies, defense laboratories

1388 students have enrolled already!!

COURSE INSTRUCTOR



Adrish received his Bachelors degree from Indian Institute of Technology, Kharagpur and Masters and Ph.D. degree from University of Notre Dame, Indiana. He is currently an Associate Professor in the Department of Electrical Engineering at Indian Institute of Technology, Kanpur. He has been a visiting faculty to National Yunlin University of Science and Technology, Taiwan and Chung-Ang University, Seoul, South Korea. Under Erasmus- Mundus program he was a visiting faculty in Politecnico di Torino, Italy. He is a recipient of Microsoft Research India young faculty award, and Institute of Engineers India young engineer award. His research interests are in the physical layer aspects of wireless communications, particularly error control coding, cognitive radio and green communications.
COURSE LAYOUT

 Week 1:
Introduction: Entropy, Relative Entropy, Mutual Information; 
Information Inequalities; 

Week 2:
Block to variable length coding-I: Prefix-free code
Block to variable length coding-II: Bounds on optimal codelength;
Block to variable length coding-III: Huffman coding. 

Week 3:
Variable to block length coding
The asymptotic equipartition property
Block to block coding of DMS

Week 4:
Universal Source Coding-I: Lempel-Ziv Algorithm-LZ77
Universal source coding-II: Lempel-Ziv Welch Algorithm (LZW)

Week 5:
Coding for sources with memory
Channel capacity of discrete memoryless channels.

Week 6:
Joint typical sequences
Noisy channel coding theorem; 
Differential entropy ; 

Week 7:
Gaussian Channel; 
Parallel Gaussian Channel.

Week 8:
Rate Distortion Theory; 
Blahut-Arimoto Algorithm for computation of channel capacity and rate- distortion function.

SUGGESTED READING

  1. James L. Massey, Lecture notes on ``Applied Digital Information Theory I''. 
  2. Thomas M. Cover, Joy A. Thomas, ``Elements of Information Theory'', 2nd Edition, John Wiley & Sons, 2006.
  3. Robert G. Gallager, ``Information Theory and Reliable Communications'', John Wiley & Sons, 1968.
  4. Raymond W. Yeung, ``Information Theory and Network Coding'', Springer, 2008.
  5. David J. C. MacKay, ``Information Theory, Inference, and Learning Algorithms'', Cambridge University Press.
  6. Robert Ash, ``Information Theory'', Dover Publications, 1965.
  7. Imre Csiszar and Jonos Korner, ``Information Theory'', Second edition, Cambridge University Press, 2011.


MORE DETAILS ABOUT THE COURSE
Course url: https://onlinecourses.nptel.ac.in/noc16_ec18
Course duration: 8 weeks
Start date and end date of course: 18 July 2016 - 9 September 2016 
Dates of exams:18 Sep 2016 and 25 Sep 2016
Time of exam : 2pm - 5pm
Final List of exam cities will be available in exam registration form.
Exam registration url - Will be announced shortly
Exam Fee: The online registration form has to be filled and the certification exam fee of approximately Rs 1000(non-Programming) needs to be paid.

CERTIFICATE

E-Certificate will be given to those who register and write the exam. Certificate will have your name, photograph and the score in the final exam. It will have the logos of NPTEL and IIT Kanpur.

It will be e-verifiable at nptel.ac.in/noc.