So, how does this work? Does the camera simply short out the flash when it fires,
Yes, it shorts the primary on the ignition coil (transformer). The current through the contacts is related to the size and charged voltage of the trigger capacitor and the impedance of the ignition coil primary windings. The older units I'm familiar with don't have a SCR to isolate this circuit.
Also known as a thyristor as I used in my description earlier.
Think of it as a diode which (as you may know, only conducts in one direction) which does not conduct until it is triggered by a voltage on a third terminal. It continues conducting until the voltage drops to (near) zero.
A variation of this is a triac which is similar in operation but conducts both ways and can be used with AC. A triac is more likely to be used for light dimming than an SCR/thyristor.
So who is to blame that some flashes put out a voltage which can damage some cameras? It is a matter of bad flash design or bad camera design? Surely it has to be one or the other, and it seems that the latter is more likely. Why don't all cameras incorporate something like the Wein device?
But it is not as much a matter of bad this or that, but more of evolution.
The old* mechanical thingies did not mind at all how many Volts were run through them. The only thing that could get damaged was the photographer holding the thing.
So why would flash manufacturers need to limit the voltage?
Cameras evolved into thingies that came to mind how many Volts they are exposed too. Flash units that take that into account evolved alongside.
So can you say that it is bad design that, say, carbide is of no use to me today when i find the headlight on my bike needs fixing?
Should the lamp manufacturer have made it so that it can still burn carbide without being damaged? Or should the chemists of the carbide supplier have thought up a new type of carbide that would be perfectly usable in modern bicycle lamps?
Or should i just go find a replacement bulb for the thingy?
*Still plenty of them around that do not mind. Old, and new.
In the days of bulb flashes, a capacitor would be charged by a battery and then the shutter contacts would discharge that into the bulb setting it off. The capacitor was used as the battery on its own would have too high an internal resistance, limiting current, which could not guarantee an instantaneous (or nearly) firing.
These contacts had to be fairly robust to handle the current.
Then along came early electronic flash with high trigger voltages. Luckily the robust mechanical contacts of the shutters of the time were also able to handle the high voltage.
For a while, camera design evolved and the mechanical contacts remained allowing the cameras to work with older flash equipment.
One day, someone must have decided to use an electronic trigger instead and to save money (probably) limited its voltage rating (to 6 volts ?) making it incompatible with most older flashes. Perhaps it was a way to sell new flashes as only the new types would be compatible.
A good design would have put high voltage capable circuitry in the camera. Imagine a Wein Safe built into the camera instead of used as an accessory. This would have been easy and use a few cheap components but some designers decided not to do it.
Actually what I was referring to was that, for example, a slave strobe will be fired by a SCR (silicone controlled rectifier) which will serve the purpose of activating the ignition transformer or coil, rather than physical contacts in a shutter. The triggering voltage for the SCR will be small, so if one put the SCR circuit between the contacts in a shutter and the ignition circuit, the contacts in the shutter would be protected. Here is an example, further isolated with a opto-coupler.