I've only been to Wales once, but it seemed like every exit from the highways had a sign for a castle. My favorite place wasn't a castle (though all that I saw that have been mentioned were great). I absolutely loved Tintern Abbey. I want to go back there so badly.
Hope you have fun!