My immediate reaction is that there is something wrong with the battery.
You can test this by plugging the unit into a strictly charging only device, such as those USB wall chargers that come with phones.
If all is well, the boot loader should detect no USB host and boot directly into Rockbox. Once RB is running, disconnect the charging cable. If it powers off immediately, I'd be convinced it is a battery issue.
What does the device do if you attempted to power it up while holding the volume down key?
There are numerous other issues to explore but we'll need a plethora of more information before that can happen. Did it work before, what lead up to this failure, is the .rockbox folder preent and intact, etc.?
BTW, if the device enters USB mode while connected to a PC your songs are still intact and can, at the very least, be rescued for use on another device.
Finally, RB for Fuze+ is still in development and not stable, it would be premature to A:] pass judgement on it as of yet B:] rely on it as your only player option.