Rust is actually the oxidization of metal and we usually want to either remove the rust from the metal itself, or remove stains that the rust left in rugs and other materials.
There are a wide variety of cleaners on the market for cleaning off rust, some of which are very toxic and often remove much more than just the rust, so always read the labels and test the product in an unseen corner first.
For removing rust from metal, a wire brush is a common and effective tool. One little tip is when you are working with a wire wheel in an electric drill, like for cleaning off that wrought iron railing, reverse the direction of rotation every 30 seconds. That stands the wires back up and attacks the rust aggressively rather than just polishing it.
One surprising success was that on straight metal, your standard soda drink took off rust every bit as well as did any of the more expensive products. (Hate to think what it does to our stomachs.)
The environmentally friendly cleaners generally work a bit slower than the aggressive acids, but of course are not dangerous to work with, and tend to be kinder to rugs and other tissues, Rust Gun being one of the best in that category.
Some metal paints are designed to be painted right over very light rust; they actually dissolve the rust and make it part of the finish. They won't dissolve large flakes but do allow you to obtain a good paint job even though you cleaned the first part of the railing a week ago and it has started to rust again.
For more information see Removing Rust Stains.