Yep, I'm exactly like you. I also have a CS degree, but after many years of doing web dev. I feel stupid. I have recently started buying books that teaches algorithmics in a more easy approach.
I am currently reading "Python Algorithms: Mastering Basic Algorithms in the Python Language". I can't recommend it enough. Besides clearly explaining the basics, it helps you learn how to think about transforming naive solutions in to efficient algoritms.
Besides that, I find it fun to practice on code tests provided by codility, interviewstreet etc.
This book is great. I would also suggest Bruno Preiss' site, which has his data structures and algo book that target specific languages (one in ruby, one in java, one in python, etc) available for free. link: http://www.brpreiss.com/
I am currently reading "Python Algorithms: Mastering Basic Algorithms in the Python Language". I can't recommend it enough. Besides clearly explaining the basics, it helps you learn how to think about transforming naive solutions in to efficient algoritms.
Besides that, I find it fun to practice on code tests provided by codility, interviewstreet etc.