Posted by: atri | September 17, 2007

Lecture 9: Converse of Shannon’s theorem

Today we proved the “negative” part of Shannon’s capacity theorem for the Binary Symmetric Channel. In particular, we showed that for every 0\le p<1/2 and 0 < \epsilon \le 1/2-p, there cannot be reliable transmission with rate 1-H(p)+\epsilon over BSC_p. If you have any questions regarding the proofs we did today, please feel free to use the comments section.

In the next lecture, we will use the probabilistic method to prove the “positive” part of Shannon’s capacity theorem for BSC_p.

Update: Someone asked me about a few steps that I skipped in the proof of the converse of Shannon’s theorem and I realized it might be better to post the missing steps here in case someone else also had the same doubt.

In the proof I implicitly used the following fact: given \mathrm{Pr}[\overline{A}]\le p_1 and \mathrm{Pr}[\overline{B}]\le p_2, \mathrm{Pr}[A|B]\ge 1-p_1 -p_2. (In the proof p_1=1/2, p_2=2^{-\Omega(\epsilon^2 n)} and A is the event \mathbf{y}\in D_m and the event B is \mathbf{y}\in \mathcal{S}.)

The above follows from the following facts (which you can convince yourself are true):

  • \mathrm{Pr}[A|B]\ge \mathrm{Pr}[A\cap B].
  • A\cap B=A\setminus \overline {B}.
  • \mathrm{Pr}[A\setminus \overline{B}]\ge \mathrm{Pr}[A]-\mathrm{Pr}[\overline{B}].

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: