My 80 had a clunk which behaved exactly like you describe. Turned out to be worn splines on the rear driveshaft, causing enough rotational play to clunk on change of direction under load. It was diagnosed and repaired by Bill at Bill's Englewood Driveshaft with a driveshaft rebuild. Hasn't repeated since.
67 FJ40 LC Tan
94 RN101 PU Pewter
95 FZJ80 LC Red
99 UZJ100 LC Black
00 VZN185 4Runner Highlander Jade
00 VZN185 4Runner SR5 Desert Dune