If you don't use a smart charger, you *will* destroy the cells from overcharging. Temperature protection is one factor only; you need to cease charging at the correct time, which is when the cell voltage reaches its peak and then drops by 5mV. If charging doesn't cease then, the cells will be reduced in capacity and the pack will have a short life. If you charge at a low rate (12-hour), the cells should barely warm at all and that will give them the longest life. If you charge rapidly (1-hour rate), the cells will heat a lot, their life will be shortened and they will receive only a fraction of their full charge before hitting 37C.
You can build a higher-voltage smart charger to charge the whole pack but for balancing reasons, it's much much better to do it cell-by-cell. Since you're already into hacking up battery packs, I suggest you put a little 8-pin plug in there that allows you to wire the four cells individually into a 4-cell smart charger with a cable. The pack will last a lot longer and hold a lot more energy.
If I were building a battery pack now, I'd use 18650 lithiums.