According to Graflex’s Pacemaker Speed and Crown Graphic Service Manual and Parts Manual, (January 1965) the ground glass has a polished side and a frosted side and, “outside” means toward the viewer, the opposite is toward the lens: "Reassemble the focus screen (7) with the polished side of the focusing screen to the outside", and "Assemble the Ektalite field lens [fresnel lens] underneath the glass with the grooved side toward the frosted side of the ground glass" (p.6 sec.6). The same thing is said concerning the back for the Super and Super Speed Graphic in Graflex Service manual (March 1965, p 19 sec.8).
This holds, as far as I understand, only for those backs to the Graphics that are constructed for the Ektalite fresnel lens. I also have a somewhat older Speed (at least I think it is older than my Crown) though it has a Graflock back the only way to attach a fresnel lens is on top of the polished side of the ground glass, the frosted side of the ground glass toward the lens (like I do on my Sinars). If you change this in some way the film plane will not coincide with the sharp picture on the ground glass (at least it should not coincide if everything is correct!) If you turn the grooved side of the Ektalite lens toward the lens it will not change the focus and you will not see the fresnel rings using the loupe, but the peculiar fresnel effect of lightening the ground glass will not be really as good (my experience at least).