ECE 788 – Network Information Theory


Description

This course covers fundamental concepts in multi-user and network information theory. The first part of the course is devoted to the introduction of basic tools and ideas such as random coding, binning, superposition coding, joint typicality decoding and capacity converses. These concepts are applied to the study of a number of source and channel models: multiple-description source coding, distributed source coding, broadcast and multiple access channels, relay and multi-relay channels, channels with feedback. Coding over networks for single-source and multi-source multicast problems is also discussed from an information-theoretic standpoint. Throughout the course, applications are presented, including the case of wireless networks.

Prerequisites

Basic knowledge of random signal analysis and linear algebra are required.

Instructor

Dr. Osvaldo Simeone
Email: osvaldo.simeone@ njit.edu
Phone: (973) 596-5809
Office: 101 FMH Building 
Office Hour: Wednesday 4-6pm and any other time by appointment

Textbooks

Topics in multi-user information theory
G. Kramer
now Publishers Inc., 2008

Related readings:

R. W. Yeung, Information theory and network coding, Springer, 2008

C. Fragouli C. Fragouli and E. Soljanin, Network coding applications, now Publishers Inc., 2007.

C. Fragouli and E. Soljanin, Network Coding Foundamentals, now Publishers Inc., 2007.

A list of references and links.


Requirements

There will be weakly assignments (30%), one midterm (30%), and one final exam (40%).


Tentative schedule

Date

Topic

Chapter covered

Aug. 31

Basic definitions, Types, Information Measures

Kramer, 1

Sept. 14

Typical sequences, Lossless source coding theorem

Kramer, 2

Sept. 21

Rate distortion and Multiple description problem

Kramer, 2

Sept. 28

Capacity-Cost

Kramer, 3

Oct. 5

Distributed source coding

Kramer, 4

Oct. 12

Rate distortion with side information

Kramer, 5

Oct. 19

Midterm

 

Oct. 26

Coding for channels with state

Kramer, 6

Nov. 2

Broadcast channel

Kramer 7

Nov. 9

Multiaccess Channel

Kramer 8

Nov. 16

Relay Channel

Kramer 9

Nov. 23

Relay Channel

Kramer 9

Nov. 30

Network Coding

 

Dec. 7

Network Coding

 

Dec. 14

Final

 


Exams and Grades

Midterm 2009: Text and solutions, Grades

Final 2009: Text and solutions, Grades

(grade mapping after final – 6-7: C+ ,7-8: B, 8-9: B+, 9-10: A).


Useful material

Lecture 1:

Csiszar, I., "The method of types [information theory]," Information Theory, IEEE Transactions on , vol.44, no.6, pp.2505-2523, Oct 1998
URL: http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=720546&isnumber=15554

 

Assignment

 

Lecture 2:

Assignment

 

Lecture 3:

P. Cuff, H. Permuter and T. Cover, “Coordination capacity,” submitted.

 

Assignment

 

Lecture 4:

Goyal, V.K., "Multiple description coding: compression meets the network," Signal Processing Magazine, IEEE , vol.18, no.5, pp.74-93, Sep 2001

 

Assignment

 

Lecture 5:

R. Koetter, M. Effros and M. Medard, On a theory of network equivalence, in Proc. ITW 2009.

 

 

Assignment

 

Lecture 6:

 

Assignment

 

Lecture 7:

Jafar, S., "Capacity With Causal and Noncausal Side Information: A Unified View," Information Theory, IEEE Transactions on , vol.52, no.12, pp.5468-5474, Dec. 2006

Heegard, C.; Gamal, A.E., "On the capacity of computer memory with defects," Information Theory, IEEE Transactions on , vol.29, no.5, pp. 731-739, Sep 1983

Cover, T.M.; Mung Chiang, "Duality between channel capacity and rate distortion with two-sided state information," Information Theory, IEEE Transactions on , vol.48, no.6, pp.1629-1638, Jun 2002

 

Assignment

 

Lecture 8:

T. M. Cover, ``Broadcast channels,'' IEEE Trans. Inf. Theory, vol. IT-18, pp. 2-14, January 1972. (pdf)

 

P. P. Bergmans, ``Random coding theorem for broadcast channels with degraded components,'' IEEE Trans. Inf. Theory, vol. IT-19, pp. 197-207, March 1973. (pdf)

 

P. P. Bergmans, ``A simple converse for broadcast channels with additive white Gaussian noise,'' IEEE Trans. Inf. Theory, vol. IT-20, pp. 279-280, March 1974. (pdf)

 

R. G. Gallager, ``Capacity and coding for degraded broadcast channels,'' Probl. Pered. Inform., vol. 10, no. 3, pp. 3-14, July-Sept. 1974. (pdf)

 

A. El Gamal, ``The capacity of a class of broadcast channels,'' IEEE Trans. Inf. Theory, vol. IT-25, pp. 166-169, March 1979. (pdf)

 

Assignment

 

Lecture 9:

T. M. Cover, ``An achievable rate region for the broadcast channel,'' IEEE Trans. Inf. Theory, vol. IT-21, pp. 399-404, July 1975. (pdf)

A. El Gamal and E. C. van der Meulen, ``A proof of Marton's coding theorem for the discrete memoryless broadcast channel, IEEE Trans. Inf. Theory, vol. IT-27, pp. 120-122, January 1981. (pdf)

Bike Xie, Richard D. Wesel, Optimal Encoding Schemes for Several Classes of Discrete Degraded Broadcast Channels, submitted.

Assignment

 

Lecture 10:

Assignment

 

Lecture 11:

Aleksic, M.; Razaghi, P.; Wei Yu, "Capacity of a Class of Modulo-Sum Relay Channels," Information Theory, IEEE Transactions on , vol.55, no.3, pp.921-930, March 2009.

El Gamal, A.; Mohseni, M.; Zahedi, S., "Bounds on capacity and minimum energy-per-bit for AWGN relay channels," Information Theory, IEEE Transactions on , vol.52, no.4, pp.1545-1561, April 2006

Y.-H. Kim, “Coding techniques for primitive relay channels,” in Proc. Forty-Fifth Annual Allerton Conf. Commun., Contr. Comput., Monticello, IL, Sep. 2007

 

Assignment

 

Lecture 12:

List of references and links on network coding

Assignment