David Mackay's Information Theory, Inference and Learning algorithms doesn't cover everything, but it ties up a lot of important topics.

http://www.inference.phy.cam.ac.uk/mackay/itila/