ENSC 861 - Source Coding in Digital
Communications
Jie Liang
Course Description
This course
covers the source coding parts of the information theory and their
applications in image and video coding. The course
first introduces the topics of entropy, information, channel capacity
and
rate-distortion functions. Various techniques used in source coding,
such as entropy coding, quantization,
and transforms are then
discussed. Some latest developments such as distributed source coding and compressed sensing will
also be introduced. The JPEG2000 and H.264 coding standards will be studied in
details.
Prerequisites
ENSC 802 or equivalent.
Mailing List
The course will have a mailing list ensc-861@sfu.ca, which you will be
able
to use to send
time-critical announcements to everyone in the class.
Grading Policy
Grading will be done according to the following scheme (tentative):
- Assignments: 20%
- Midterm exam: 25%
- Final Exam: 35%
- Project: 20%
All exams are closed-book and closed-note. No formula sheet is allowed for the midterm exam. Calculators and two pages of
letter-size (8.5 x 11 inch), hand-written
and double-sided formula sheets
are allowed for the final exam (No PDA and cell phones please). You must show all
important work on an exam to receive
full credit.
Assignment Information
There will be approximately 5 homework assignments in this course and
they
will be returned approximately two weeks after they are submitted.
Depending
upon the TA support provided by the Department, it is possible that
only a
randomly chose subset of the problems will be graded.
Note that some important concepts will be covered only in the homework
assignments.
Plagiarism Issues:
Please review the following page on plagiarism when you work on your
project:
http://www.lib.sfu.ca/researchhelp/writing/plagiarism.htm
Pay attention to the following forms of plagiarism:
- Quoting material without proper use of quotation
marks (even if otherwise cited appropriately)
- Using art, graphs, illustrations, maps,
statistics, photographs, etc. without complete and proper citation
- Paraphrasing or summarizing information from a
source without proper acknowledgment
You should state clearly in your report whether you have used any
source code from other people, and what modification you have made to
improve it, if any.
As in any other course, anyone caught cheating on the project report
will receive an automatic F.
The Text Books
There will be no required text book
for this course. Most of the
materials of the course will be pulled from the following books, which
are reserved in the library:
- T. M. Cover,
Elements of Information Theory, Wiley, New York, 1991.
- David S. Taubman,
Michael W. Marcellin, JPEG2000:
image compression fundamentals,
standards, and practice, Kluwer Academic Publishers, Boston,
2002.
- Khalid Sayood. Introduction
to Data Compression. 2nd
Edition. Morgan Kaufman. 2000 (The third edition was published in Dec.
2005 and will be available in SFU library soon).
Click
here for the Companion Web Site for this book: A good place to find
some useful sample codes.
- N. S. Jayant and
Peter Noll, Digital Coding of
Waveforms: Principles and Applications to Speech and Video,
Prentice Hall, Englewood Cliffs, NJ, 1984.
- A.Gersho, R.
M.
Gray, Vector Quantization and Signal Compression, Kluwer Academic
Publishers, Boston, 1992.
- G. Strang, T. Q. Nguyen, Wavelets and Filter Banks,
Wellesley-Cambridge Press, Wellesley, MA, 1997.
- P. P. Vaidyanathan, Multirate systems and filter banks, Prentice
Hall, 1992.
Syllabus
The following is a brief synopsis of what will be covered in this
course. Note that this material is still evolving.
- Information theory fundamental:
- Entropy, relative
entropy, mutual information, and differential entropy
- Entropy
Coding:
- Shannon's source coding theorem
- Huffman coding
- Golomb-Rice coding
- Arithmetic
coding
- Rate Distortion Theory
- Gaussian source
- Correlated Gaussian source
- Generalization to 2-D signals (image, video, multiview video)
- Quantization:
- Scalar quantization
- Vector
quantization
- Trellis coded quantization
- Transform Coding:
- KLT, DCT
- Lapped
transform
- Wavelet and filter banks
- Multimedia coding standards:
- JPEG
- JPEG-LS
- JPEG XR
- JPEG 2000
- H.264
- Advanced Topics:
- Optimization in source coding
- Joint source channel coding
- Distributed source coding
- Hieg efficiency video coding
