Enh -- empirically I think "fine art" photography is whatever the guy behind the camera decides to call fine art. I usually use quotes around the phrase to indicate I don't take it too seriously. One or two posts upthread mentioned sales - I don't see that as a requirement either. I've seen work that got awards, presumably based on composition, lighting, tones and other factors, yet the subject was some rusty machinery, crumbling building or the like that didn't have customers lining up with checkbooks in hand!
I also agree that some may use it to imply non-commercial, non-contract, 'I do what I feel like doing' photography, which is pretty much how I see it. I actually got a call from someone who'd seen and liked my work that wanted an event photographed -- had to say "sorry, don't do that."
I'd say "just do it." Eventually you may hit upon some inspiring series that has others calling you a "fine art" photographer. Then it's gotta be true!