At some point you have to bite the bullet and do that major price hike and get into the prestige gallery. There's a reason they're still in business with a brick-and-mortar retail operation in SoHo or Midtown in Manhattan - they're making the sales to cover their rent and then some. Those galleries will only take you on if they KNOW they can sell your work to their clientele, and for prices that will make both of you money. Just as a gallery has no obligation to accept a given artist, the artist is not bound by some devil's pact signed in blood and the artists' first-born child to any particular gallery, if the gallery doesn't perform for the artist. Good galleries will prompt you to make and show them new work on a regular basis, and will work with you. Since galleries are not much more than ad agencies for their photographers, they're really only as good as their last show.
If in the end it means you lose the smaller market galleries and your "affordable" sales there, then you do that. Maybe you don't do that now, at only four years in the "art" business, but at some point you'll have to, unless you want to intentionally limit your sales and your income. That's a value-call for you to make, and not one that anyone else can make for you, or for anyone to judge you by.