I have had this problem in the past. I did NOT have the problem with the Sansa OF, and did not originally have it when I RBed the Fuze+. I think it is specific to Fuze+, as my older Fuze V2 never had the problem.
The problem started for me about RB 3.10 or 3.11, I think.
The problem was gone with 3.12, although I think I was getting some low-level background noise, on some screens (scanning the disk, etc.).
I haven't used the Fuze+ for some time and after charging it, when I booted in order to verify this problem, I was getting all sorts of noise - squeeks and squawks, on all screens, and in playback (which was just noise).
I re-booted into Sansa OF, and all was quiet. Just took forever to load, which was one of the reasons I RBed in the 1st place.
I download/installed the latest Dev RB (130201) and all is quiet now. Also, still (for me) no pops on startup.
Hope this helps. Try the latest Build.