If you've got dirty or scratched glass then of course you'll get defects in your prints. It stands to reason. If you want to use a sheet of diffuser (which in effect turns your enlarger into a softbox) then that’s great, do it. But there’s really no need.
It’s well established by many, many master printers that a sheet of glass is necessary to keep the negative pressed flat against the paper. In my own experience I’ve seen soft spots on a print where there hasn’t been complete contact – and that’s when I’m using a printing frame. In fact the third scan you posted on the LF forum shows exactly what I’d expect if you’re negative doesn’t have glass (and as an aside the middle “glass & diffuser” scan suggests to me your negative isn’t completely flat under the glass anyway). I expect that your diffuser will actually be making the blurriness worse in this case (softer light). And removing the glass is not going to eliminate dust/hair defects (e.g. the right hand defect in your first scan) because that stuff is just as likely to settle on your negative or your paper.
Using a clean and unscratched piece of glass that’s either heavy enough to flatten the paper on its own or is clamped to the paper in some way (e.g. a contact printing frame) will fix the blurriness because your negative will be flat against the paper. It will fix the shadows because there will be no scratches to cast shadows (remember I said “clean and unscratched”). And if you clean the paper, negative and glass before you make the sandwich then you’ll only have to worry about dust on the surface of the glass (which you’d have to worry about anyway without the glass).
What I don’t understand is why you reject using glass when most of your problems seem to be caused by the specific piece of glass you’re using. And I don’t understand why, after having the patience to restore a large format camera you don’t have the same patience when it comes to learning how to contact print. My suggestion is that you should do what countless others have done before: find out how the master printers do it and then copy their process. That’s the quickest way to creating the stunning prints you’re aiming for.