You pay a premium for realty and property taxes, but in some ways NJ is really cheap. There is no sales tax on clothes, gas is very inexpensive as well as food. The thing about car insurance is a bit of an exaggeration, I have lived in 4 different states and car insurance is maybe 10-20% more than anywhere else.
For durable goods you always get a discount from list, which you often get stuck paying elsewhere.

The train that runs from philly-newark-new brunswick-new york is very fast, I would definitly consider commuting somewhere in the corridor. I have a cousin living just north of Trenton, very charming area on the delaware with old slate-roof houses (affordable no less). You may want to consider renting and getting a lay of the land before you buy anything.