Posted by: atri | September 21, 2007

Lecture 11: Shannon vs. Hamming

In today’s lecture, we redid the last part of Shannon’s proof for the capacity theorem for BSC_p. We then looked at the questions that were left open by Shannon’s proof and compared them to the corresponding questions in Hamming’s world. We then did a quick comparison of the salient points of Shannon’s and Hamming’s theory. We then started to look at some bounds on the rate of a code (as a function of the relative distance). At the end of the class we proved the Singleton bound. Next class, we will look a particular family of codes called Reed-Solomon codes that meet the Singleton bound.

In the lecture, I mentioned the following paper: Error-free coding by Elias. This paper gives a very hands down construction of a code with rate R>0 that can be used for reliable communication over BSC_p for some p>0. You might also want to look at the lecture notes from Venkat Guruswami‘s coding theory course that talks about this paper. I encourage you to read the paper: the construction is very nice and we have covered most of the bakcground needed for the result. I also encourage you to read the introduction of the paper: it’s from an era where introductions were written in a different manner than today 🙂


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: