It's in southwest Georgia. Right now is a good time to go, summer the heat in the canyon is almost unbearable. Heres a link:
http://gastateparks.org/info/providence/

Theres quite a few wild places to go in Georgia, just like most any other place, you just have to know how to find them. Fort Mountain up north by Chatsworth has a lot of interesting stuff around it, Cloudland Canyon towards Trenton in the N.E. corner of the state has some nice vistas and hiking trails.
Savanah has some really nice acrhitecture and nearby Fort Pulaski is a lot of fun to shoot old brick at.