That's about it. From the standpoint of optics, there's not too much one can do to damage ground glass outside of outright breakage. The main reason not to wipe is to insure absence of fibers inside the camera where they might easily be transferes to film later.
I've used cameras where the gg was cracked, "rubbed" (gg smoothed due to "contact" with sharp object), or otherwise damaged. There is a consideration of resale value or preservation of course.
R. Eugene Smith