I could see it messing with a mouse, wired or wireless. They don't exactly use the best of shielding and suppression on a $10 throw-away mouse. Still, an HT can definitely generate a greater field strength than a cell phone, even at max power (where cell phones rarely run anyway). Your cell phone might generate a 4 V/m E-field within a meter of the phone but a 5 W HT even with a modest gain antenna can generate a 20 V/m field within a meter. This is a considerable EMI field, although not terribly harsh.
OTOH, I don't see how an ATM card would care. A mildly strong electric field of 20 V/m in free space of 377 ohms and generates a B-field of roughly 53 mA/m. This gives you an H-field around 0.7 milligauss. To erase a magnetic stripe /quickly/ would take quite a bit stronger H-field. I think those bulk card erasers produce in the range of 10,000 A/m B-fields (e.g. 125 gauss).
As an comparison, standing a few feet under typical household power lines will subject you to about 25 milligauss of magnetizing field. An MRI is around 1 tesla or about 1,000 gauss. The magnetic seal on your fridge or the magnet on car speakers are probably in the 1 to 10 gauss range. The Earth's own field is around 500 milligauss. In fact, I'd guess that residual magnetism on a pocket knife or the speaker in the radio are more likely culprits than any generated E-field.
'91 Pickup - Imelda
'08 Tacoma TRD - Donna
'09 Kawasaki KLR650
'12 Gunnar Rockhound 29
"Our task must be to free ourselves from this prison by widening our circle of compassion to embrace all living creatures and the whole of nature in its beauty. Nobody is able to achieve this completely, but the striving for such achievement is in itself a part of the liberation and a foundation for inner security. " --Albert Einstein