The mat color is all part of the great mystery of What "Works" and What Doesn't. I've been pondering that one for some time, now, and I don't think I'm any closer to the answer than when I started.
I remember Herb Ritts' show at the MFA in Boston. All the work had black, canvas textured mats ... and It all seemed to WORK. That show populated five (5) LARGE rooms in the museum.
If anything, I tend (n.b. "tend") to match the "key" with the mat. A high-key image seems to work well with a white mat; low-key with black....
I do have a Nude in brilliant autumn foliage matted with a black mat, and that particular combination has an uncanny amount of "depth". It seems to rival an image viewed through a stereoscope.
Two of my gallery images here will be hung in a local show tomorrow: Shagbark Hickory ... in a black mat; and Classical Nude, in white.