This is a subject that has been discussed extensively on the web - both by those who are relatively knowlegable and by those who are not burdoned by any factual information. In fact, there is even a forum that is devoted exclusively to discussion of the rights of photographers, and how those rights are being challenged inappropriately by both government officials and the general public.
I don't believe that it is inherently illegal to photograph a post office. But given today's paranoia, it would not be unusual to be challenged. This is especially the case if you are photographing a major post office in a large city, and even more if the building also contains other government offices and courts.
In the US, you can legally photograph anything you can see while standing on the sidewalk or other public property. Of course you have to be careful about what constitutes public property - shopping malls, for example, are private property.
There are a few notorious instances where local jurisdictions have created regulations against photographing of certain public facilities. In New York, there are signs on all bridges declaring that photography is illegal, and the New York Transit Authority wants to make photography illegal in the subways. It remains to be seen whether those ban will be held up on appeal. In some places (New York City and Washington, DC, for example) it is illegal to use a tripod on public property without a permit - this is supposed to be in the interest of public safety or to control potential public nuisance - frankly, there is some validity to that argument, but it is also possible that it is being carried to a ridiculous extreme.
There are those who believe that the Patriot Act includes provisions against photographing certain public buildings, bridges, etc. I haven't done the research myself, but I saw a posting by one individual who claimed to have studied the Patriot Act and found that there is no reference to photography AT ALL in that law.
The best available information on what is legal (and conversely, what is not) in this area can be found on Bert Krages web site, http://www.krages.com/lvaserv.htm.