Lakhasly

Channel coding, initiated by Shannon's 1948 paper, aimed to achieve practical coding schemes reaching the Shannon limit on channels like the AWGN channel. This proved difficult, but the past decade saw success with turbo and LDPC codes. This paper recounts this journey, focusing on the AWGN channel. Section II reviews the Shannon limit definitions. Section III examines algebraic coding, its achievements, and limitations in reaching the Shannon limit. Section IV explores the "probabilistic coding" approach, starting with Elias' convolutional codes, and encompassing advancements like concatenated codes and trellis decoding, leading to modern capacity-approaching codes. Section V discusses codes for bandwidth-limited channels (lattice codes and trellis-coded modulation). Finally, Section VI details the development of capacity-approaching codes, primarily turbo and LDPC codes.

The field of channel coding started with Claude Shannon’s 1948 landmark paper [1]. For the next half century,
its central objective was to find practical coding schemes that could approach channel capacity (hereafter called
“the Shannon limit”) on well-understood channels such as the additive white Gaussian noise (AWGN) channel.
This goal proved to be challenging, but not impossible. In the past decade, with the advent of turbo codes and the
rebirth of low-density parity-check codes, it has finally been achieved, at least in many cases of practical interest.
As Bob McEliece observed in his 2004 Shannon Lecture [2], the extraordinary efforts that were required to
achieve this objective may not be fully appreciated by future historians. McEliece imagined a biographical note in
the 166th edition of the Encyclopedia Galactica along the following lines:
Claude Shannon: Born on the planet Earth (Sol III) in the year 1916 A.D. Generally regarded as the
father of the Information Age, he formulated the notion of channel capacity in 1948 A.D. Within several
decades, mathematicians and engineers had devised practical ways to communicate reliably at data rates
within 1% of the Shannon limit . . .
The purpose of this paper is to tell the story of how Shannon’s challenge was met, at least as it appeared to us,
before the details of this story are lost to memory.
We focus on the AWGN channel, which was the target for many of these efforts. In Section II, we review various
definitions of the Shannon limit for this channel.
In Section III, we discuss the subfield of algebraic coding, which dominated the channel coding field for its first
couple of decades. We will discuss both the achievements of algebraic coding, and also the reasons why it did not
prove to be the way to approach the Shannon limit.
Daniel J. Costello, Jr., is with the Univ. Notre Dame, IN 46556, USA, e-mail: [email protected].
G. David Forney, Jr., is with the Mass. Inst. of Tech., Cambridge, MA 02139 USA, e-mail: [email protected].
This work was supported in part by NSF Grant CCR02-05310 and NASA Grant NNGO5GH73G
In Section IV, we discuss the alternative line of development that was inspired more directly by Shannon’s
random coding approach, which is sometimes called “probabilistic coding.” The first major contribution to this
area after Shannon was Elias’ invention of convolutional codes. This line of development includes product codes,
concatenated codes, trellis decoding of block codes, and ultimately modern capacity-approaching codes.
In Section V, we discuss codes for bandwidth-limited channels, namely lattice codes and trellis-coded modulation.
Finally, in Section VI, we discuss the development of capacity-approaching codes, principally turbo codes and
low-density parity-check (LDPC) codes..