I do see a separation, since we stayed with Witkin I will use his work as example. In it he uses props etc, to adjust his "vision" a documentary photo lacks this. Usually is one where the subject if photographed with flat lighting to show maximum detail and there is no consideration to "artistic" merit.
There is no reason why a documentary photo cannot become an art photo but the intial intention is very clear and it shows in the print. For example I am sure all remember the photo of the vietnamese "spy" which was executed and the photographer took the shot right at the moment the person pulled the trigger. That is a documentary shot where the "shock" is secondary to the oeverall picture. In Witkins case, the shock is the priary purpose and the props and lighting are used to maximaze this effect.
As you stated in the case of the museum and the circus freaks, there is in all of us a morbid curiosity to see the unusual. I dont think this constitutes the basis for lasting art. In any case to each its own, personally I rather have nice things to look at hanging on my wall than mutilated and deformed bodies.