My Toro LX424 mower drive belt pops off frequently. It came off yesterday, just driving out of the garage and later after I had mowed for an hour or so (I had to push the mower home).
The belt was new last season and does not appear to be badly worn or stretched. The tension on the belt appears to be pretty good and the tension spring seems to be fine, too.
I replaced the variable speed belt from the motor last week.
You don't say how old the unit is, but I suspect that your idler tension pulley may be worn or that there may be belt fingers which may be bet or out of alignment.
I checked the tension idler pully. It didn't appear to be worn, but it was a little wobbly. I tightened it down and that seemed to take care of the problem. I had loosened the nut last season in order to change the belt. I must not have tightened it enough.