So much depends on what you want. Seattle is BIG, and the urban sprawl goes a long way. Victoria is smaller, and you can get out of it in less time. But that's me - when I'm in a city my first concern is getting out of it again. There's always something to see at the Butchart Gardens, but November is not going to give you the punch of colour you see in their advertising. Now Goldstream Park, just outside Victoria has the salmon run this time of year - which also means lots of scavenging birds including eagles. Then there's the big ol' trees all over the place, hiking trails, waterfalls, the estuary, Mount Finlayson, not to mention you can camp at Goldstream all year round. (We spent Christmas there in our little camper last year.) Fort Rodd Hill (and Fisguard Lighthouse) and Royal Roads, just west of Victoria, are very easy to access by bus. They're also great for wandering around with a camera.
I've spent very little time in Seattle, and just wanted to get the heck out when I was there. No offence Seattlites - I feel the same in ANY big city - Vancouver, Tokyo, Hamburg. If you like big cities, the advice to head to the centre is probably the best.
It is the Pacific coast in November - typically the greyest, wettest month. Be prepared.