A good introduction to information theory, please?

Question

I know about Wikipedia and MacKay Information Theory, Inference and Learning Algorithms (Does this fit like a tutorial?). We study a textbook starting with Shannon's entropy and passing through conditional entropy and mutual information ... Any idea? If you conduct such a course at your university, what textbook is used?

Thanks.

Answer 1

I used the following tutorial while studying at CS in EPFL . IMO is well written, with good explanations and covers more than enough for an introduction to the subject area.

Elements of Information Theory

EDIT: For further reading, here are some other readings that my professor recommended. I did not read them (shame on me), so I can’t say whether they are good or not.

1. R. G. Gallager, Information Theory and Reliable Communications , Wiley, 1968.
2. D. Mackay, Information Theory, Inference, and Learning Algorithms , Cambridge University Press, 2008. (You have already mentioned this)
3. I. Csiszar and J. Korner, Information Theory: Coding Theorems for Discrete Memoryless Systems, Chiado Academy, 1997.
4. S. E. Shannon, Mathematical Theory of Communication