Just back from 2nd trip this year, 5 months total. In HCMC there is an old pagoda in Cholon (in many of the guidebooks) with some interesting photo ops. The Cao Dai cathedral in Tay Ninh (day trip from HCMC) is a feast. Be sure to go upstairs. Floating market at My Tho has lots, and anywhere in the countryside, especially late afternoon, has many opportunities.
Don't sweat security unduly. Just be prudent and especially careful near tourist sites and foreigner hotels.