That's a fairly primitive view of photography. It is true that photography wasn't taught in art schools or studied in art history programs until relatively recently (1960s or so), but that's not really the case any more.

I wouldn't worry about it, really. Just do the work, and let the critics sort out what it is.