It will only boot the Apple firmware if you have the hold switch turned on; this is how dual booting works, see the manual. If you leave the hold switch off, it should boot Rockbox when you plug it in.
The Apple firmware is not stored on the data partition; deleting the other directories is perfectly safe but they will be recreated if you boot the original firmware. You can always restore the iPod to stock using iTunes, it doesn't care what you've done to it.
There's currently no easy way to make Rockbox boot even if the hold switch is turned on: even if you install Rockbox directly in place of the Apple firmware, eliminating the dual boot option, then what you'll find is that booting Rockbox with the hold switch enabled instead erases all your settings, since turning hold on *during* boot is how this is normally done. To actually do this safely you need a custom build of both the bootloader and Rockbox itself, which removes the dual boot and settings-reset functionality.