I agree completely with Bob on this one. While it's good to choose the most flattering angle for the subject, the more important factor, for me, is determining what is most effective for the mood / tone you want to set. If you were photographing a powerful CEO, you'd probably not choose to shoot downward at him, as you don't want to make him look doe-y eyed and innocent. On the other hand, if you were doing a soft, feminine portrait of a woman and shot from a downward angle, you'd be exaggerating the jaw and making the eyes look smaller, which is rarely what a female subject has in mind.
It's not about being sexist, it's about choosing the angle that best suits the purpose of the image.