Family of 5 will fit in a 100 better than an 80, esp. if there are child seats involved. This may be your deciding factor.
I love my 80 but it is no longer my daily driver, so I don't have to worry about MPG or road manners. If I did, I might be driving a 100 series myself or a recent vintage 4Runner.