ECE 776 - Information Theory

Description

This is a course on information theory and its applications at graduate level. Topics include basic concepts and definitions of information measure; asymptotic equipartition property and its applications; theory of data compression; and definition and theory of channel capacity. Advanced topics that also will be addressed include maximum entropy and spectral estimation; relationship between information theory and statistics; rate distortion theory; network information theory; and information theory for wireless communication. 

• Information Measure: Entropy, mutual information, information distance, and their relationship.

• Asymptotic equipartition property: AEP and its consequences. 

• Data Compression: Kraft inequality, data compression codes.

• Channel Capacity: Definition, Shannon coding theorem, Gaussian channel capacity. 

• Information Theory and Statistics: Method of type, large deviation theory. 

• Rate Distortion Theory: Definition, inverse water-filling.

• Network Information Theory: Multiple access channel, broadcast channel, relay channel.

Prerequisites

Basic knowledge of random signal analysis at the level of ECE 673 is required. Certain amount of analytical skill and mathematical maturity are also necessary for the course.

Instructor

Professor Roy You
Email: you@adm.njit.edu
Phone: (973) 596-3528
Office: 333 ECE Building 
Office Hour: Monday, Tuesday 4-5pm.

Textbook

Elements of Information Theory
Thomas M. Cover and Joy A. Thomas
John Wiley & Sons, Inc. 1991.

Grading Policy

There will be weekly problem sets (30% of grade), one midterm (40% of grade), and one project (30% of grade). The project and some of the problem sets might involve Matlab simulation. You can obtain a copy of Matlab software from the campus computing facility. 

Schedule