Wouldn't it be too large? Maybe there are other, smaller SBCs out there that can be used (preferably one with battery terminals).
But if you can get past that, it can be done. You'll need to deal with HDMI however since it doesn't have a DSI (for video, there is one for camera but no idea if it works with displays) connector. EDIT: It does composite as well apparently.
There's also the power supply problem. Since it only takes USB power, you'll need a separate voltage converter, and charging circuitry. A powerbank does this, but the circuit adds bulk and is ultimately inefficient.
Not sure how good hifiberry is, but it may suffer from interference (from the RPi).
Rockbox can be ported as an app running on some embedded linux.
Also the new RPi0W adds wireless comms, allowing BT and wifi.