Caulking on the outside of the house has very little to do with stopping cold air drafts from getting into the house. It's purpose is to weather proof the wall against rain penetration. It doesn't generally stop air flow since most siding is purposefully ventilated.
Caulking inside the house is used to seal off holes and cracks that are not designed to move, like door frames, window frames, floor boards and electrical outlets, stopping air from moving either in or out of the house.
Weather-stripping is used to stop air flow, in or out, through joints between things that are designed to move, like openable windows and doors.
What most people miss in their air sealing efforts is that the air leaks on the top of the house are the most important, but the least noticed. These cracks allow warm moist air to flow into the walls and attic of the house where they can cause condensation, ice and rot. Always start your air sealing efforts at the top of the house and work your way down. That way you deal with the cold air drafts that got you started on all of this work only after you have taken care of the more important ones that are bothering your house.