The viewfinder has to be smaller by definition. It has to have a moving mirror. That means its dimensions fit inside the normal "view" dimensions inside the mirror chamber...

It's just one of the issues with splitting a view off in a separate direction (the viewfinder) from where it needs to go naturally (the film plane).