I could see it messing with a mouse, wired or wireless. They don't exactly use the best of shielding and suppression on a $10 throw-away mouse. Still, an HT can definitely generate a greater field strength than a cell phone, even at max power (where cell phones rarely run anyway). Your cell phone might generate a 4 V/m E-field within a meter of the phone but a 5 W HT even with a modest gain antenna can generate a 20 V/m field within a meter. This is a considerable EMI field, although not terribly harsh.
OTOH, I don't see how an ATM card would care. A mildly strong electric field of 20 V/m in free space of 377 ohms and generates a B-field of roughly 53 mA/m. This gives you an H-field around 0.7 milligauss. To erase a magnetic stripe /quickly/ would take quite a bit stronger H-field. I think those bulk card erasers produce in the range of 10,000 A/m B-fields (e.g. 125 gauss).
As an comparison, standing a few feet under typical household power lines will subject you to about 25 milligauss of magnetizing field. An MRI is around 1 tesla or about 1,000 gauss. The magnetic seal on your fridge or the magnet on car speakers are probably in the 1 to 10 gauss range. The Earth's own field is around 500 milligauss. In fact, I'd guess that residual magnetism on a pocket knife or the speaker in the radio are more likely culprits than any generated E-field.
"Silent enim leges inter arma." -- Cicero