For books and magazines, the vertical format is just easier from a production and handling point of view; horizontal books are harder to hold. Books have been made that way since the invention of the codex. For art that's hung on a wall, I suppose that's not so much of an issue.
John Logie Baird adopted a portrait format for the first video recordings (late 20's) and broadcast television (early 30's).
What's a 'natural' field of view?
Do you make postcard-sized prints and look at them from (a) 6 in/15cm or (b) 12 in/30cm; or 12x16 inch/30x40cm prints and view them from (a) 12 in/30cm (b) 24 in/60cm or (c) 36 in/90cm?
Consider the angle subtended by the subject, at the eye, at the time of taking, and the angle subtended at the time of viewing the print. When the two match reasonably closely there is a particularly three-dimensional effect.
Our department secretary has a screen that rotates, and she seems to leave it in vertical mode.
There are a few cameras that are natively vertical, like the Bronica 645 RF and the Linhof 220 (there was a horizontal version, but it's much rarer than the vertical one), and I'd suspect some of the long roll portrait cameras.
I work in finance but a few years ago moved from the analytical side to the legal documentation. I now find it more frustrating working from a monitor since, as you note, it favors spreadsheets over text!
Alas, being a lowly attorney, I do not have the clout of an Admin Asst. to insist on a rotatable screen. Heck, in our shop, I'm happy if the damned system doesn't crash!
BTW: I do find, in 35mm format, I prefer the vertical with, of course, the exception of the standard landscape shot.
A relatively cheap video card will allow you to choose the orientation of the screen, the screen itself needn't have a rotation feature. Simply get one of the cards (I think most any nVidia card from a 7600GS up will do it) and mount the montor sideways.
To introduce anything into your computer at work - even putting a CD into the slot or attaching a data stick - is grounds for employment termination!
Great idea for freelancers and consultants, however. ;)
I am all too aware of the strictures of corporate weaseldom having been a weasel of the boogle once and possibly always.