Yes...you could definitely build an in-line chiller. Whether it would be efficient or not depends on a lot of variables, though. There are challenges. Actually I did this several (maybe 15) years ago. I had the idea of incorporating a water cooler into a mini fridge or incorporating a mini-fridge into a water cooler, depending on your point of view. A salesman I knew at Market Hall had one of those around 1961, and I thought it was a really neat combination. You can see what I came up with below. The heat exchanger side of the device was made of brass and I made a serpentine-like affair that fed water in one end and out at the other. It worked but the trick was feeding water through slowly enough to extract heat and then storing the cold water until you needed it. What I ended up deciding was that the best arrangement would be a copper canister which would hold a quart or more and sit inside a typical refrigerator with a supply line going in and a delivery line going back out. That will work, but you have some plumbing challenges and the delivery line must be insulated if you are planning on bringing it back to a faucet as it will condense water around here like crazy. That is essentially what a fridge that serves water from the front door does. It might be something best done in a house which had a wet bar (which was equipped with a mini-fridge or an ice machine). At least that way the delivery line wouldn't have far to go.
Jon