You can always overmat if you don't like the brush strokes. I find messing around with drafting tape to be more hassle than I'm willing to do usually so I just do brush strokes (although as neat and tidy as possible). I kind of like the dark border on some things as it's easy to "hide" signature/title in there using a pencil.
Also some people use rod coating which tends to be a little "neater" looking.
I say do whatever makes you happy. There's no rules either way and as you've noticed, sometimes it's good and sometimes it's bad for certain images.