alfers is right. reverse osmosis is the best. You can get a small unit that runs to your main faucet for drinking purposes or a water line that supplies a refrigerator. The filters can clog up fast if the water is really bad and you just replace the filters that usually screw into the unit.
I don't think i'd waste my time filtering the entire house.