Slides, exercises, and hints for the exercises are uploaded here as they become available.
13 January 2014: Uncertainty and Inference
Probability distributions
Random variables
Conditional probability
Slides; exercises; hints.
The Hartley measure of uncertainty
Brute-force coding
Shannon entropy
Huffman coding
Slides; exercises; hints.
Random variables
Conditional probability
Slides; exercises; hints.
The Hartley measure of uncertainty
Brute-force coding
Shannon entropy
Huffman coding
Slides; exercises; hints.
14 January 2014: Counting Typical Sequences
Variance
The law of large numbers
The central limit theorem
Slides; exercises; hints.
Typical sequences
The source coding theorem
Slides; exercises; hints.
The law of large numbers
The central limit theorem
Slides; exercises; hints.
Typical sequences
The source coding theorem
Slides; exercises; hints.
15 January 2014: Random Processes
Random processes
Stationary distributions
Ergodic processes
The general source coding theorem
Slides; exercises; hints.
Arithmetic coding
Slides; exercises; hints.
Stationary distributions
Ergodic processes
The general source coding theorem
Slides; exercises; hints.
Arithmetic coding
Slides; exercises; hints.
16 January 2014: Guessing and Gambling
Kullback-Leibler divergence
Variational approximation
Codebreaking
Likelihood ratios
Slides; exercises; hints.
Doubling rates
Proportional betting
Slides; exercises; hints.
Variational approximation
Codebreaking
Likelihood ratios
Slides; exercises; hints.
Doubling rates
Proportional betting
Slides; exercises; hints.
17 January 2014: Explained and Unexplained Randomness
Kolmogorov complexity
Encoding of numbers
Kolmogorov complexity and coin flipping
Slides; exercises; hints.
Estimation
The bias-variance tradoff
Slides; exercises.
Encoding of numbers
Kolmogorov complexity and coin flipping
Slides; exercises; hints.
Estimation
The bias-variance tradoff
Slides; exercises.