Was the magnetometer a DC or AC unit, was it an analog or digital type and was it touching (or very, very close to) the antenna? A value of 5 G sounds very high to me and I suspect that your magnetometer can't handle the frequency of an RF EM field. Could also be an interference problem, since generally they're not really designed to work in the presence of high RF.
None-the-less, most are accurate up to maybe 100kHz for AC magnetic fields and are typically spec'd at 60Hz. Finding an RF magnetic field is done analytically by measuring the electric field, which is far easier to do accurately at RF. Once you know E-field strength then it's just a matter of applying Maxwell's Equations. That is how I arrived at my expected magnetic fields above, by starting with a 20V/m E-field and working forward. Lots of assumptions, materials for one make a difference on how an incident field affects you.
Now, I don't know if those are realized but I do know that an HT drives a lot less current (you'll see around 300mA into a dummy load) than a 120VAC overhead line and so I would expect as relative distances they would have similar magnetic fields. IOW, 10 feet from a 1800W household circuit strikes me as more similar to a foot from a 5W RF transmitter than not. Several orders difference seems counterintuitive.
Washing machine numbers do not surprise, motors are all about electromagnetism. It's not transformers as such, but current that generates an EM field. The physics of how transformers work necessitate they must generate large fields. Just a coincidence that things with xfrmrs also have high power. Your truck battery shorted with a huge wire will also make a electromagnet, as is the case, for example, inside your alternator and starter for the field windings.
"Why does the U.S. care which flag will be hoisted on a small piece of land thousands of miles away?" -- Ron Paul