This has to do with the fact that rockbox isn't boosting the cpu when no music is playing, so when you're scrolling at the time, it's using less cpu power to process it, herego slower scrolling. When music is playing the cpu is boosted so it's running at more power, and scrolling is quicker.
That said, this thread doesn't belong on this board...sorry for replying, Llorean!