Oh, I somehow skipped this thread. Maybe I could copy what I already replied as a PM so that everyone can read it (there's nothing new here, but repeating things I've learned at APUG):
Indeed, I used glyoxal. I'm not sure whether I tried formalin too, but glyoxal has been my hardener of choice. As per PE's suggestion, I dilute the commercial 40% solution by 1+9 to 4%, and then use this at 5 ml per 100 grams of emulsion.
In the most successful PET coating, I had only minor frilling at cut edges - something less than 1mm frills.
I found that the sorbitol is almost a must, otherwise it is too "hard" and curls with incredible force. The sorbitol solved this problem like a magic. I can check the amount I used, I should have it somewhere down...
My ultimate recipe included one or two small drops [per 20 ml of emulsion] of Photo flo equivalent (I am using Agfa Agepon), the glyoxal and sorbitol.
I have to add that I add the hardener to a really warm emulsion. The emulsion is somewhere at about 40-42 deg C (105 F). And, stir/mix well!