Today we stared with the proof of the converse of Shannon’s theorem and then moved on to the proof of the “positive” part of Shannon’s theorem. The relevant lectures notes from Fall 07 are Lecture 9 and Lecture 10 respectively. (I’ll polish the latter notes by this weekend.)

In class today, I stumbled during the proof of a claim, which I promised to prove on the blog. The claim was that . It turns out that I can prove the following weaker claim, which nonethless is sufficient for the proof: , for some small enough . (We’ll also have to consider a “tighter” shell than the one we defined in class today.) The notes for Lecture 9 from Fall 07 has been updated with the correct proof.

Thanks to Luca Trevisan‘s latex2wp program, I’m also reproducing the proof of the converse of Shannon’s theorem after the fold. (Please let me know if you find any bugs!)

**1. Converse of Shannon’s Capacity Theorem for BSC **

Theorem 1Let and . If then for every and , forsome,

*Proof:* First, we note that there is nothing to prove if , so for the rest of the proof we will assume that . For the sake of contradiction, assume that the following holds for *every* :

Fix an arbitrary message . Define to be the set of received words that are decoded to by , that is,

Note that by our assumption, the following is true (where from now on we omit the explicit dependence of the probability on the noise for clarity):

Further, by the Chernoff bound,

where is the shell of radius around , that is, . (We will set in terms of and at the end of the proof.)

(1) and (2) along with the union bound imply the following:

where the last inequality holds for large enough . Next we upper bound the probability above to obtain a lower bound on .

It is easy to see that

where

It is easy to check that is decreasing in for . Thus, we have

Thus, we have shown that

which by (3) implies that

Next, we consider the following sequence of relations:

In the above (5) follows from the fact that for , and are disjoint. (6) follows from (4). (7) follows for large enough and if we pick . (Note that as , .)

(7) implies that , which is a contradiction. The proof is complete.

Remark 1It can be verified that the proof above can also work if the decoding error probability is bounded by (instead of the in the theorem) for small enough .

## Leave a Reply