Do you have a soft top or just a bikini top (or nothing)? If you have a soft top, you can wait until around Turkey Day. If you've got nothing or a bikini top, it's going to start to get colder at night. We usually get the first big snow around Halloween.
I've been running the soft top exclusively for the past 4 years, but my 40 isn't my daily driver right now. If it was my DD, I'd probably put the hard top back on around Nov. 15.