Information Theory Assignments
- 1st Week
-
- Read Oishi (textbook) Chapter 1
-
- Mathematica Exercize: set union, intersection, complement (difference)
-
Submit on A4 paper at the next class;
don't forget to include your name and student id at the top.
Refresh:
Refer to Toyoizumi-sensei's useful notes
from Computer Literacy lectures
for reminders about Mathematica.
- Use Mathematica to calculate set union, intersection, and complement (difference)
using digits from your student ID as sample data.
- Hint: Use something like
list = IntegerDigits[ToExpression[StringDrop[$UserName,1]]]
to generate sample data.
Use something like lists = Partition[list, 4, 3]
to break that list into two lists.
Use Intersection[]
(or ESC inter ESC, where "ESC" is the "Escape" key), Union[]
(or ESC un ESC), and Complement[]
.
(Use this sample session as a model if you get stuck.
[You can save this file through your browser
(right-click or control+click in Firefox to "Save Link As...",
right-click in Safari to "Download Linked File As...",
right-click in Explorer to "Download Link to Disk",
etc.)
to your file system,
then open in Mathematica.])
- Extra credit
-
What is the set of (unique) digits in your student ID?
That is, the answer will have the set of the digits in your student number,
with each digit appearing only once.
For instance,
the set of digits in "s1181234" is {1,2,3,4,8}
(and not {1,1,1,2,3,4,8}).
(Hint: convert from a list to a set by monadically (using a single argument) invoking one of the set operations on a list.)
- Mathematica Exercize: average & variance
-
Write Mathematica functions to calculate mean (average) and variance. Use the digits
of your student ID as sample data, as you did on the previous assignment. Print out onto A4 paper and attach to the first part of the assignment.
Hint: Use the functions
Apply[Plus, data]
and Length[data]
, along with arithmetic functions -
(subtraction) and ^
(exponentiation), to calculate these two values. Experiment with the Mathematica palettes for expression entry (of, for example, fractions and powers).
(Use this sample session
or this method for calculating Pi from random numbers
if you need reminders about Mathematica syntax.)
- Bring in Y1,000 to buy Chinese Rings puzzle.
- 2nd Week
-
- Do Problem 1.3 (p. 14 of Oishi, repeated on p. 19 of course notes).
Note that E(X)2 is equivalent to E2(X) (but not E(X2)).
Hand in solution on A4 paper.
- Read Oishi 2.1-4
- Play with "Patience Puzzle" (a.k.a. "Chinese Rings"). (Double extra credit if you can demonstrate its solution.)
- 3rd Week
-
- 4th Week
-
- Play with "Patience Puzzle." (Extra credit if you can demonstrate its solution.)
- Study for 1st midterm exam (open book, open notes, calculator and dictionary okay).
- 5th Week
-
- Do Oishi problems 2.2, 2.5. (Don't forget to record your name and student number at the top of the sheet.)
- Do the problem on p. 48 of the lecture notes, corresponding to Oishi 2.6:
Draw code trees for C7 and C8, calculate average code length, and confirm the Kraft Inequality.
- Solve the Patience Puzzle!
Demonstrate its solution to get credit.
(Explore the course reference links for hints if you need.)
- 6th Week
-
- (No new assignment.)
- Solve the Patience Puzzle!
Demonstrate its solution to get credit.
(Explore the course reference links for hints if you need.)
- 7th Week
-
- Read Oishi sections 3.1-3.6.
- On an A4 piece of paper,
- Do the exercizes on p. 64 of the lecture notes. Hand in on A4 paper. (Don't forget to write your name and student ID at the top.)
- Study for 2nd midterm exam.
- 8th Week
-
- Read Oishi sections 3.6-3.9
- Hand in on A4 paper [don't forget to write your name and student id]
- Do problem on p. 82 of the course notes (linked to Section 3.8, p. 53, of Oishi): make a Huffman code based on probabilities (16/32, 8/32, 4/32, 4/32).
What happens when accumulated probabilities that equal some other probability are sorted above the previous value?
- Calculate entropy H(1/3, 1/3, 1/3). Use of of Mathematica, calculator, etc. encouraged. (Don't forget to use Log[2, whatever] in Mma to get binary logs.)
- Calculate H(1/3, 1/3, 1/6, 1/6)
- 9th Week
-
- 10th Week
-
-
- Mathematica Exercize: Markov processes
-
Use Mathematica to calculate steady state (eigenvector) of a Markov process (reflecting a matrix made from your student ID):
- Use the sample session as a model. (Right- or control-click to download.)
- Replace the sample student ID at the top with your own.
- Compile the sample code to make a unique transition matrix modeling an ergodic Markov process and find its eigenvector both iteratively and analytically. (Reminder about Mma: Edit>Select All to select the entire file, then Shift+Return to execute.)
- Save the compiled Mathematica file as concatenation of your student ID and "-Markov" with the Mathematica file type extension ".nb" (for example, "s1181234-Markov.nb").
- Email it as an attachment to TA Prabath at m5141110@u-aizu.ac.jp by Monday morning.
- 11th Week
-
- (No assignment; Happy New Year!)
- 12th Week
-
- Do exercize in section 4.5.2.1 of notes (p. 101, '11-'12 edition).
- Read Oishi 5.
- 13th Week
-
- 14th Week
-
- Read the Appendix in Oishi (about Analog/Digital analysis).
-
Determine the 6th roots of 1,
expressed in polar (degrees and radians) and rectangular (complex) notation.
Sketch these roots on a unit circle.
(submitted on A4 paper)
- Study for final exam
Michael Cohen