Peter in Gander NF has the opposite problem of water hammer. When he turns the hot water ON in the shower, he gets a loud bang in the plumbing. Water Hammer is where you get a bang when you turn the water off.
Peter's problem is usually caused by a washer type faucet that has a loose washer, usually the screw in the end completely missing. It could be that the water flow in this faucet is such that the water pressure holds this loose washer shut even after the faucet shaft has been backed off, and then suddenly it pops open. It could also be simply a very old washer that is mechanically stuck on the seat, the shaft pulls back and then suddenly the water pops through.
In either case, simply replace the washer and make sure it has a screw in the end to hold it in place. By the way, while the faucet is open, flush some water through the valve and very likely that missing screw will spit itself out.