Posted by: atri | September 19, 2007

Lecture 10: Shannon’s theorem

Today, we say the proof of Shannon’s theorem for BSC_p that states that for every 0\le p<1/2 and 0<\epsilon\le 1/2-p, there exists encoding and decoding functions such that reliable transmission is possible for all rates less than 1-H(p+\epsilon).

We proved the result using the so called random coding with expurgation argument:

  1. We picked a random encoding function (and used the MLD function) and showed that the expected average error probability is small. This implies that there exits an encoding function for which the average (over the messages) decoding error probability is small.
  2. For the decoding error to be small for all messages simultaenously, we sorted the messages by their decoding error probability and threw away the top “half.”

The second step decreases the dimension of the code by 1, which does not affect the rate much. In the begining of the next lecture, I’ll go through the last part of step 2 above as I went through that part pretty quickly today.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s


%d bloggers like this: