I think it's fair ball to just get a 35mm kit just to explore your own artistic vision, or more fully explore your photographic work.
Myself, I don't shoot 4x5 often but I like having the option...so I keep my Crown Graphic. I think the same applies to a curiosity about 35mm format.
On a related note, don't be so quick to write off Leica bodies for price. Some fine quality ones out there for around a grand, sometimes less. No reason you can't be happy with an M3, or an M4 (heck I won a bit of a lottery in finding my M6 for $1K but that's another story).
Good luck with your journey.