ECE 788 – Network Information Theory


This course covers fundamental concepts in multi-user and network information theory. The first part of the course is devoted to the introduction of basic tools and ideas such as random coding, binning, superposition coding, joint typicality decoding and capacity converses. These concepts are applied to the study of a number of source and channel models: multiple-description source coding, distributed source coding, broadcast and multiple access channels, relay and multi-relay channels, channels with feedback. Coding over networks for single-source and multi-source multicast problems is also discussed from an information-theoretic standpoint. Throughout the course, applications are presented, including the case of wireless networks.


Basic knowledge of random signal analysis and linear algebra are required.


Dr. Osvaldo Simeone
Email: osvaldo.simeone@
Phone: (973) 596-5809
Office: 101 FMH Building 
Office Hour: Wednesday 4-6pm and any other time by appointment


Topics in multi-user information theory
G. Kramer
now Publishers Inc., 2008

Related readings:

R. W. Yeung, Information theory and network coding, Springer, 2008

C. Fragouli C. Fragouli and E. Soljanin, Network coding applications, now Publishers Inc., 2007.

C. Fragouli and E. Soljanin, Network Coding Foundamentals, now Publishers Inc., 2007.

A list of references and links.


There will be weakly assignments (30%), one midterm (30%), and one final exam (40%).

Tentative schedule



Chapter covered

Aug. 31

Basic definitions, Types, Information Measures

Kramer, 1

Sept. 14

Typical sequences, Lossless source coding theorem

Kramer, 2

Sept. 21

Rate distortion and Multiple description problem

Kramer, 2

Sept. 28


Kramer, 3

Oct. 5

Distributed source coding

Kramer, 4

Oct. 12

Rate distortion with side information

Kramer, 5

Oct. 19



Oct. 26

Coding for channels with state

Kramer, 6

Nov. 2

Broadcast channel

Kramer 7

Nov. 9

Multiaccess Channel

Kramer 8

Nov. 16

Relay Channel

Kramer 9

Nov. 23

Relay Channel

Kramer 9

Nov. 30

Network Coding


Dec. 7

Network Coding


Dec. 14



Exams and Grades

Midterm 2009: Text and solutions, Grades

Final 2009: Text and solutions, Grades

(grade mapping after final – 6-7: C+ ,7-8: B, 8-9: B+, 9-10: A).

Useful material

Lecture 1:

Csiszar, I., "The method of types [information theory]," Information Theory, IEEE Transactions on , vol.44, no.6, pp.2505-2523, Oct 1998




Lecture 2:



Lecture 3:

P. Cuff, H. Permuter and T. Cover, “Coordination capacity,” submitted.




Lecture 4:

Goyal, V.K., "Multiple description coding: compression meets the network," Signal Processing Magazine, IEEE , vol.18, no.5, pp.74-93, Sep 2001




Lecture 5:

R. Koetter, M. Effros and M. Medard, On a theory of network equivalence, in Proc. ITW 2009.





Lecture 6:




Lecture 7:

Jafar, S., "Capacity With Causal and Noncausal Side Information: A Unified View," Information Theory, IEEE Transactions on , vol.52, no.12, pp.5468-5474, Dec. 2006

Heegard, C.; Gamal, A.E., "On the capacity of computer memory with defects," Information Theory, IEEE Transactions on , vol.29, no.5, pp. 731-739, Sep 1983

Cover, T.M.; Mung Chiang, "Duality between channel capacity and rate distortion with two-sided state information," Information Theory, IEEE Transactions on , vol.48, no.6, pp.1629-1638, Jun 2002




Lecture 8:

T. M. Cover, ``Broadcast channels,'' IEEE Trans. Inf. Theory, vol. IT-18, pp. 2-14, January 1972. (pdf)


P. P. Bergmans, ``Random coding theorem for broadcast channels with degraded components,'' IEEE Trans. Inf. Theory, vol. IT-19, pp. 197-207, March 1973. (pdf)


P. P. Bergmans, ``A simple converse for broadcast channels with additive white Gaussian noise,'' IEEE Trans. Inf. Theory, vol. IT-20, pp. 279-280, March 1974. (pdf)


R. G. Gallager, ``Capacity and coding for degraded broadcast channels,'' Probl. Pered. Inform., vol. 10, no. 3, pp. 3-14, July-Sept. 1974. (pdf)


A. El Gamal, ``The capacity of a class of broadcast channels,'' IEEE Trans. Inf. Theory, vol. IT-25, pp. 166-169, March 1979. (pdf)




Lecture 9:

T. M. Cover, ``An achievable rate region for the broadcast channel,'' IEEE Trans. Inf. Theory, vol. IT-21, pp. 399-404, July 1975. (pdf)

A. El Gamal and E. C. van der Meulen, ``A proof of Marton's coding theorem for the discrete memoryless broadcast channel, IEEE Trans. Inf. Theory, vol. IT-27, pp. 120-122, January 1981. (pdf)

Bike Xie, Richard D. Wesel, Optimal Encoding Schemes for Several Classes of Discrete Degraded Broadcast Channels, submitted.



Lecture 10:



Lecture 11:

Aleksic, M.; Razaghi, P.; Wei Yu, "Capacity of a Class of Modulo-Sum Relay Channels," Information Theory, IEEE Transactions on , vol.55, no.3, pp.921-930, March 2009.

El Gamal, A.; Mohseni, M.; Zahedi, S., "Bounds on capacity and minimum energy-per-bit for AWGN relay channels," Information Theory, IEEE Transactions on , vol.52, no.4, pp.1545-1561, April 2006

Y.-H. Kim, “Coding techniques for primitive relay channels,” in Proc. Forty-Fifth Annual Allerton Conf. Commun., Contr. Comput., Monticello, IL, Sep. 2007




Lecture 12:

List of references and links on network coding