My suggestion of Nimbin was very much tongue in cheek, and I expected when the OP got there he'd take a note of the ambience and whilst finding it interesting, drive on.
I wouldn't live there either. But there are plenty of nice small towns in the border regions to choose from without having to embrace any particular culture. Anyway, most of the hippies have gone mainstream and now work at regular jobs for the Council in Lismore. There are a few die-hards around though.