Paging requires a
memory management unit, and most targets don't have that.
If paging was used actively, the hard drive accesses would spin up the hard drive a
lot and use a lot of power.
What might make sense instead is if Rockbox had various modules which were loaded from disk. For example, there might be recording, settings, playlist editor and playback modules which are loaded when needed. However, like Llorean says, there is plenty of RAM, so this probably wouldn't be worth it. Even on HWCODEC targets with 2 megs you probably wouldn't gain much.