Hagerman isn't much worse than a dirt road, but if there is snow (and mud) present it could be dangerous if not impassable. Call the forest service (White River I think) and find out the condition of the pass and the snow conditions.
Other than that, the drive on the west side along the Fryingpan River is spectacular - a perfect way to get to Aspen.
1976 FJ40 - green and mostly stock
1996 FZJ80 - not so stock