several years ago, i took a class from prof tibshirani using this book and was absolutely blown away. if you don't have a deep stats background (as i didn't), you might find it scarily dense, but wow there's some good stuff in there.
I also used this book for a data mining class that I took about two years ago. The book is very thorough but the math can be dense. We didn't actually end up using it in class very much, but it was a good reference book for the topics it contains.
I am starting to learn statistics. But I have zero knowledge in the subject. I have picked up a copy of Headfirst Statistics. Is that a good one? Any recommendations?
"All of Statistics" by Larry Wasserman is very good. It is traditional, fairly mathematical statistics, but intended for a general audience. (It is particularly popular with computer science people.)
It is also extremely concise, perhaps too much so occasionally for pedagogical ease. It really does come pretty close to covering "all of statistics" in 442 pages, an impressive feat. But as you can imagine it certainly doesn't beat around the bush much. So if you're looking for something with multiple applications, covering the same material from different perspectives, etc., or just want more hand holding, this may not be for you.
