I have submitted the grades: you should be able to view it through myUB. If you have any questions (e.g. breakup of your scores for different parts of the course), please let me know.

It was great having you all in the course. I will be posting somewhat infrequently on this blog for the rest of the year, so come back and check the blog out once in a while.

I was asked about this so here is a rough breakdown of the different phases of the wikiepdia part.

1. 60% of your score will depend on the version that you turned in. 
2. 30% will depend on your final in house version
3. 10% will depend on the version you upload to Wikipedia. (There is a chance that folks might change what you have uploaded, I’ll check the version you put up, so let me know what your user name is so that I can track your version.)

Please use the comments section, if you have any questions on this.

Here are some important dates regarding various stuff in the course:

1. Lecture notes: Any lectures notes or revisions should be turned in by Tuesday, May 5 midnight Wednesday, May 6 midnight.
2. Wikiepdia entries: I’ll get back to you with comments on your entries by Wednesday, April 29. The deadline to revise your entry will be Wednesday, May 6 midnight  Sunday, May 10 midnight.  You will upload your entries onto Wikipedia by Friday, May 8 at 8pm  Tuesday, May 12 at 8pm. In other words, I expect you to take ownership of your entry on Wikipedia– please let me know if this is a problem.

First, I’d like to take you out for lunch. The choices are noon-12:50 on either Thursday, May 7 or Friday, May 8. Unfortunately, I can only do noon-12:50 on Friday, May 8. Please use the comments section to let me know if you can make it.

Below is the schedule for the presentations:

• Thursday, May 7

1. 1:00-1:50pm: JD
2. 2:00-2:50pm: Jeff
3. 3:00-3:50pm: Swapnoneel

• Friday, May 8

1. 1:00-1:50pm: Steve
2. 2:00-2:50pm: Willmert
3. 3:00-3:50pm: Krishna

Hopefully, I did not mess up your preferences above. If your slot does not work for you, please use the comments section to let us know.

(Update 5/7): Just confirming that the talks on Friday will also be in Bell 242. Also we’ll meet at the Indian place at noon tomorrow for lunch.

In today’s lecture, we looked at the intuition behind why folded RS codes can achieve list decoding capacity in polynomial time. The intuitions are from this survey. Unfortunately, we did not have any time to talk about the details. You can look at the corresponding lecture notes from Fall 07 (which have to be polished). Alternatively, you can look at this chapter from my thesis.

A semester is too short a time to do justice to the numerous interesting topics in coding theory.  Let me mention some of these topics in no particular order (that we either mentioned in passing in the class or did not mention them at all).

• LDPC codes. We just saw the definition of these codes. As I had mentioned in class, these were defined by Gallager in his thesis in the 60s.The late 90s saw a resurgence in research activit in LDPC codes. These latter codes can provably achieve the capacity of the . They also experimentally seem to achieve the capacity of the . The big advantage for these codes are the linear time encoding and decoding algorithms (the dependence on , the distance from capacity, is also some small polynomial). For more details see this survey by Venkat Guruswami.
• Expander codes. In class we showed that expander codes are aysmptotically good. These give us the only linear time encodable and decodable binary codes (in the worst-case noise model). For more details on expanders codes as well as another class of application of expanders to codes, see this survey by Venkat Guruswami.
• Algebraic Geometry codes. These codes beat the Gilbert-Varshamov bound for alphabets . Steve will tell us more about these codes in his presentation. See my post on the corresponding project topic from fall 07 for more pointers.
• Linear Programming bounds. The best known lower bound on the rate vs. distance question are achieved via the so called Linear Programming bound. See my post on the project topic from Fall 07 for more pointers.
• Convolutional Codes. All the codes that we covered in class were block codes. That is, the block length of such codes are fixed. However, there are many applications where having a fixed block length might be too wasteful. Convolutional codes allow for variable block lengths and “on the fly” encoding and decoding.
• Applications in Complexity theory. There are numerous applications of coding theory in theoretical computer science and in particular, complexity theory (and cryptography). We only had time to cover two such applications: communication complexity and -wise independent sources. However, some of the biggest recent advances in complexity theory (for example the PCP theorem) have used tools from coding theory. For more details, see this survey by Luca Trevisan or the my blog posts on the following project topics: codeword testing, extractors, locally decodable codes, and hardness amplification. For applications of list decoding in complexity theory see this survey by Madhu Sudan.

In today’s lecture, we stated and analyzed the Guruswami-Sudan list decoding algorithm for RS codes and showed that it can list decode up to fraction of errors. The slides for today’s lecture have been uploaded. Also the material appears in the scribed notes for Lecture 38 and 39 from Fall 07 (both need to be polished).

At the end of the lecture, we defined the Folded Reed-Solomon codes. We have no class on Friday. In our last lecture on Monday, we will look at the intuition behind the algorithms for Folded RS codes that can achieve the list decoding capacity.

In today’s lecture, we formally presented Sudan’s list decoding algorithm for RS codes that can correct up to fraction of errors in polynomial time. The stuff we covered today appears in Lecture 38 from Fall 07 (the notes need to be polished).

Next lecture, which will be from 1:30pm-3:20pm in Commons 9, we will study the improvement to Sudan’s algorithm by Guruswami and Sudan that allows us to list decode RS codes from fraction of errors in polynomial time. Thus, RS codes are explicit codes that can be algorithmically list decoded up to the Johnson bound.

In the lecture on Monday,  April 20 we will skip a calculation. Below is the calculation in its gory details.

The main definition is that of -weighted degree of a monomial. In particular, the -weighted degree of the monomial is . The main question is the following:

Given a degree bound , how many monomials in two variables are there that have weighted degree at most ?

In other words we need to find out how many distinct tuples with positive integers exist such that ? Below we calculate a lower bound. For notational convenience, define .

It is easy to check that the number of such tuples is

Unravelling the second sum, we get

Expanding the sum above, we obtain that the required number is

Moving the common term outside, we get

Till now we have not any approximation. However, we will do so now. Note that by the definition of , we have and . This implies, that the number of monomials of -weighted degree at most is at least

which is the bound that we will use in the lecture.

In Friday’s lecture, we started with the two natural questions in list decoding:

1. Can we achieve list decoding capacity with explicit codes and efficient decoding algorithms? In particular, one we achieve for large enough alphabet size?
2. As a less aggressive goal, can we achive the Johnson bound, i.e. , with efficient list decoding algorithms.

In this course, we will see positive answers to both questions with codes that are either RS codes or are generalizations of RS codes. Towards this end, we started with list decoding algorithms for RS codes. We started with the generalization of the Berlekamp-Welch algorithm by Sudan, which leads to an efficient list decoding algorithm that can correct fraction of errors. We did not specify one crucial aspect of the algorithm, which we will do in Monday’s lecture. We looked at an example that gave the intuition behind why Sudan’s algorithm. You can find the example in this survey.

Due to limited time, we skipped an easier version of Sudan’s list decoding algorithm that can correct up to fraction of errors. For more details on this, see the scribed notes of Lecture 37 from Fall 07. (The latter notes are not polished yet: hopefully they’ll be in a better shape by the end of the week.)

In today’s lecture, we showed that an expander code based on an expander results in a binary linear code of rate at least and has relative distance at least . The crux of the argument was to show that any vector of small enough weight, which corresponds to a subset of the left vertices has at least one unique neighbor (i.e. a right vertex that has exactly one edge between itself and a vertex in ). We also mentioned (without proof) that the argument can be strengthened to prove a relative distance of . (For a proof by picture  jump to after the fold.)

We also quickly saw a natural decoding algorithm, where a left vertex flips its value if a strict majority of the parities it is involved in unsatisfied with the current bit values on the left vertices. We saw how we can again use the unique neighbor argument (for ) to argue that as long as the number of errors are at most , there is always a “flip-able” vertex. This almost completes the proof that the algorithm works, except that we need to prove that we are not “converging” to a wrong codeword. The latter fact is true but we did not have time to prove. We also did not have the time to prove that the algorithm can be implemented in linear time.

Steve raised an interesting point that the algorithm we considered in class outputs the transmitted codeword and not the actual message. I think doing it for general expander codes is hard/unknown.  (I have not thought about this for long so I could be wrong: let me know if you see a way otherwise!) However, I’m reasonably sure that in Spielmans’ linear time encodable and decodable codes based on expanders, the linear time decoding can be made to output the actual message. (Again, let me know if I’m wrong!)   is systematic and hence the corresponding decoding algorithm does also output the message bits.

All the omitted details from today’s lecture can be found in Lecture 13 of Guruswami’s coding theory course. 

We now present a “proof by picture” for the assertion that in any  expander, for any subset such that has the property that , which in turn proves that the corresponding expander code has relative distance at least as was claimed before the fold.

For the proof, note that if , then the proof we did in the class works. So for the rest of the proof assume that . Now consider the situation below:

In the figure above is of size exactly . The same colored subsets on the right are the corresponding neighborhood sets, i.e. the blue oval is and the pink oval is . Now consider the unique neighborhood of , i.e. in the picture:

Till now, we have not done anything new. However note that 

.

Next consider the neighborhood of the set , which is colored green:

Note that in the figure above, does not cover all of . So is this because I cannot draw properly or is something else going on? It turns out that this is always going to be the case. To see this, first note that

.

This implies that

.

Thus, , which implies that there is always a vertex like the yellow vertex below (that belongs to ):

Note that the yellow vertex has no neighbor in . In other words, the yellow vertex belongs to , which proves that , as desired.