This is a course on information theory and its applications at graduate level. Topics include basic concepts and definitions of information measure; asymptotic equipartition property and its applications; theory of data compression; definition and theory of channel capacity; rate distortion theory; network information theory.
“Information theory […], a unifying theory [of data compression and transmission] with profound intersections with Probability, Statistics, Computer Science and other fields, continues to set the stage for the development of communications, data storage and processing, and other information technologies.” [Sergio Verdú, see here].
Basic knowledge of random signal analysis at the level of ECE 673 is required.
Dr. Osvaldo Simeone
Email: osvaldo.simeone@ njit.edu
Phone: (973) 596-5710
Office: ECE 211
Office Hour: Wednesday 4-6pm
Elements of Information theory
T. M. Cover and J. A. Thomas
Wiley.Further reading:
Information theory and reliable communication
R. G. Gallager
John Wiley & Sons, Inc.
There will one midterm (40%), and one final exam (40%) and a project (20%) to be completed by the date of the final exam. Weekly problems will be assigned but not graded. Some problem sets might involve Matlab simulation. You can obtain a copy of Matlab software from the campus computing facility.
For the project, a recent paper will be assigned to each student based on individual interests. Twenty-minute presentations to the class on the selected subject will be scheduled by the end of the semester.
Some links
Week |
Date |
Plan |
Chapter covered |
1 |
Jan. 25 |
Introduction |
1 |
2 |
Feb. 1 |
Information Measures |
2 |
3 |
Feb. 8 |
Information Measures |
2 |
4 |
Feb. 15 |
Zero-Error Data Compression |
5 |
5 |
Feb. 22 |
AEP |
3 |
6 |
Feb. 29 |
Lossless Data Compression |
3 |
7 |
Mar. 7 |
Midterm
|
|
8 |
Mar. 21 |
Entropy
Rate |
4 |
9 |
Mar. 28 |
Channel Capacity |
7 |
10 |
Apr. 4 |
Channel Capacity |
7 |
11 |
Apr. 11 |
Differential entropy |
8 |
12 |
Apr. 18 |
Gaussian channel |
9 |
13 |
Apr. 25 |
Rate-distortion
theory |
10 |
14 |
May 2 |
Rate-distortion
theory |
10 |
15 |
|
Final |
|