Posted by: atri | September 17, 2007

## Lecture 9: Converse of Shannon’s theorem

Today we proved the “negative” part of Shannon’s capacity theorem for the Binary Symmetric Channel. In particular, we showed that for every $0\le p<1/2$ and $0 < \epsilon \le 1/2-p$, there cannot be reliable transmission with rate $1-H(p)+\epsilon$ over $BSC_p$. If you have any questions regarding the proofs we did today, please feel free to use the comments section.

In the next lecture, we will use the probabilistic method to prove the “positive” part of Shannon’s capacity theorem for $BSC_p$.

Update: Someone asked me about a few steps that I skipped in the proof of the converse of Shannon’s theorem and I realized it might be better to post the missing steps here in case someone else also had the same doubt.

In the proof I implicitly used the following fact: given $\mathrm{Pr}[\overline{A}]\le p_1$ and $\mathrm{Pr}[\overline{B}]\le p_2$, $\mathrm{Pr}[A|B]\ge 1-p_1 -p_2$. (In the proof $p_1=1/2, p_2=2^{-\Omega(\epsilon^2 n)}$ and $A$is the event $\mathbf{y}\in D_m$ and the event $B$ is $\mathbf{y}\in \mathcal{S}$.)

The above follows from the following facts (which you can convince yourself are true):

• $\mathrm{Pr}[A|B]\ge \mathrm{Pr}[A\cap B]$.
• $A\cap B=A\setminus \overline {B}$.
• $\mathrm{Pr}[A\setminus \overline{B}]\ge \mathrm{Pr}[A]-\mathrm{Pr}[\overline{B}]$.