I just tried a different cable, no luck.
I also tried a different USB port, no luck.
I think it has to do with the USB mapping that is stored in the registry or WMI somewhere. I think if I have my Sansa map to a different drive than H, it might work better. Not sure where that info is stored to try.
...
OK, so I found the registry key and deleted it and I think it got me closer to understanding the problem (hopefully some rockbox developers are looking at this).
After I deleted the reg key and plugged the unit back in, the drivers installed again as they did in the past (when I first plugged in the unit) only it seemed take longer. It took longer because it also installed the drivers for the micro SD card slot even though 1) there is no card in the slot and 2) this unit has never had a card in the slot. So something (Rockbox software or maybe the Sansa firmware) thinks my Sansa is my wife's Sansa where the micro SD card maps to the M drive. The models are slightly different as well.
I guess it isn't a big deal, I think if I install a 4 GB micro SD card it will fix the sluggishness of it connecting and transferring files.