Paul you missed my point completely. I'm not saying that we look at b&w photos with rods.
What I am saying is that most of the visual signal in low light is monochromatic and less acute. And that may affect how we interpret b&w imagery. (not how we actually see it) That's a big difference. Our visual system consists of sensors and also a very powerful and complex interpretive apparatus; the latter can override the former in some cases.
Just try to imagine a coloured version of Brassaļ's night work in Paris. Does it work for you? Why not?