If you are "not serious about photography" and "just want to slap a filter and go", then I, too, suggest yellow or none at all.
Although they are called "contrast filters", they don't actually change contrast. They filter certain color. It is called "contrast filters" because if you take images including blue sky, the will darken the sky in the order of yellow-orange-red. If you are taking flowers, color might actually interfere with your images. Especially with flowers, you'll need to understand exactly what color does what, and it contradicts with our goal with not being serious and want a one filter for all concept.
Yellow is the most mild one you found out. As you go darker, it'll start to change significantly - which might very well result in changes you don't desire.
Either that or you can buy an inexpensive set and try them out - but again, it will contradict with not serious and one filter for all concept.