I'd disagree.
You may be one of the rare people who will take into account the image and only the image, but few others will be uninfluenced by how the image is presented.
The "packaging" may also give a clue as to the overall sensibility that informs the photographer, especially now that those who are bothering to make portfolios are quite often artists using photography as a medium, rather than photographers trying to get work as photographers ... if you see the distinction I am trying to make?