One area the fast lens when stopped down might not show as good performance is in contrast. A 50mm f:1.2 lens has much more area of internal reflecting surfaces than a 50mm f:2, regardless of the actual working aperture. Multicoating helps, but no matter what you do, more area equals more internal reflection. There are also optical tradeoffs when designing a large aperture lens.