The different firmware versions on the Clip are *different actual hardware*: rockbox works on the Clip v1 hardware, which has firmware version 1.x, but it does not yet work on the Clip v2 hardware, which has firmware version 2.x. A separate Rockbox port is needed for the clipv2, which is still in very early stages and is not usable.
You cannot "downgrade" as the v1 and v2 Clip are different players; they can't run each others' firmware.