In the /.rockbox/doom/ folder, you should have the rockdoom.wad and any other games you might have (tnt.wad, doom.wad, doom2.wad...) plus saves(doomsav0.dsg...) and your tranmap.dat, .doomrc, and default.dfg plus the folder addons/ which contains the addons that you'll try to run.
if the error comes after you try to start the game, and it runs its scripts, its likely that it modifies the doom engine, and is unplayable in rockbox.
when does it give you this error?