Posted by: atri | September 19, 2007

## Lecture 10: Shannon’s theorem

Today, we say the proof of Shannon’s theorem for $BSC_p$ that states that for every $0\le p<1/2$ and $0<\epsilon\le 1/2-p$, there exists encoding and decoding functions such that reliable transmission is possible for all rates less than $1-H(p+\epsilon)$.

We proved the result using the so called random coding with expurgation argument:

1. We picked a random encoding function (and used the MLD function) and showed that the expected average error probability is small. This implies that there exits an encoding function for which the average (over the messages) decoding error probability is small.
2. For the decoding error to be small for all messages simultaenously, we sorted the messages by their decoding error probability and threw away the top “half.”

The second step decreases the dimension of the code by 1, which does not affect the rate much. In the begining of the next lecture, I’ll go through the last part of step 2 above as I went through that part pretty quickly today.