There are really only two practical methods to scientifically test for heat losses in a residential house:
Infra-red thermography: using infrared sensitive film you can take a "heat picture" of the house. Hot and cold areas show up on the infra-red film to give an image of your house -- it's more like an X-ray than a photograph. It must be done in the winter because a good picture requires strong temperature differences (like an ordinary photograph requires light and dark differences). Taken from the outside of the house it will show all the conductive heat losses through the wall (lack of insulation) and all the hot air leaks. Taken from the inside of the house it will show all the cold air leaks coming in. Infra-red thermography is particularly useful for identifying the unknown condition of an existing home, and for verifying the installation of loose-fill insulation in the walls. It is expensive!
Pressure testing: As a large percentage of heat losses are due to air leaks, pressure testing your house can identify how many air leaks you have and where they are. It is usually done by replacing your front door with a giant fan and sucking the air out of your house. A mini-computer on the fan analyzes the air leaks. Before-and-after pressure testing is a standard part of a professional all-house sealing job