The rough areas of cities would be about the only place I'd be really concerned.
There are lots of wilderness places where people do mix with wildlife, as with very few beach goers around the world encountering sharks, most people go there and come back with no problem. Doesn't mean you shouldn't take reasonable precautions though.
In actuality the same is true for the not so great parts of cities, but I'd put more trust in critters than fellow humans in many places. Mostly the critters prefer to avoid you, which isn't always the case otherwise.
I've photographed from helicopters and small planes, no worries there (IME anyway).
Besides some parts of some cities, my major no-go places would be in the path of destructive storms like tornados or hurricanes. Generally they are not so difficult to avoid though, especially if you are already mobile.