Ok cool thanks for the advice! I did start looking at a CRF250x which I think would be pretty suitable with a little bit of lowering. I have avoided lowering bikes in the past because I was afraid it might mess up handling, but I would be cool with taking it to a good shop to have them dial it in because of my weight and short inseam.
I'll definitely post up here when I get a bike since all the guys I know are into strictly street and I'll be looking for people to ride with!
Current: 2014 200 series
Former: 1998 100 series