Matt, I wouldn't waste a lot of money on a fancy multimeter for automotive applications. The $10 ones I have been buying from Harbor Freight work just fine, and they're cheap enough that you can keep one in the glovebox of every vehicle like I do. And if you run it over or lose it, no big deal. They are plenty accurate for automotive work.
The description that Robbie gives above is the way I have been diagnosing bad alternators for 30 years, and has been 100% accurate. My experience is a bit different in that most vehicles that I have worked on will read 13.8 to 14.2 volts even at idle, provided the only load is the ignition system, however I have seen a few that were only in the mid 13s. Anything under 13.0 at idle means something's wrong. And as he suggested, turn on all your loads and see what you drop to.
An alternator has 3 internal diodes (rectifiers). One of the common failure modes is that these diodes fail, however I have seen a few cases where one or two fail but there is at least one or two good ones. When this happens, the alternator can not provide as much current as it is supposed to, so when you draw a fair amount of current, you're actually drawing it from the battery. That's why it's important to check voltage with loads being drawn. Most of the time, when an alternator is completely shot, the battery starts out at maybe 12.5V or so, then you start up and it drops down to 12.3 or 12.2 or so. This is the vehicle running on battery power only. Other, less common failure modes include partially shorted windings, mechanical failure, and worn brushes.
An inexpensive hygrometer can help you determine what kind of shape your battery is in. Most regular (non-optima) batteries only last 4-6 years, and they hold less and less of a charge as they age. While cold weather makes battery problems obvious due to increased internal impedance, it is high temperatures which wears them out. Overcharging exascerbates wear from high temperatures.
My offer to have an auto-related basic electronics class for the club, similar to Matt's GPS presentation, is still standing. We would need about an hour to cover the basics: ohms law, using multimeters, common auto electrical problems and diagnostic techniques.