In California, a family making low six-figures is upper-middle-class (my parents made ~$80k and we were generally considered lower-middle-class). So it makes sense to me not to call that "rich," though of course the value of money is a personal thing.
You're right that location changes the income levels quite a bit. I would need much more in California or New York to live as though I felt rich enough.