Good morning, BVY;
Radio Shack is an obvious source for new pieces.
Goodwill, Saint Vincent de Paul, Deseret Industries, The Salvation Army Thrift Stores, and other second-hand stores often will have used computer mice, and various headphones and earbuds that will have some kind of a phone plug. If there are ones there that are broken (No, don't do that!), they might just hand it to you. Just make sure that the plug is the right diameter for your microphone input jack or socket on the sound card in the computer.