Yes. If you're using the File Browser, from the root directory press Menu and select "create playlist." The resulting playlist will be called "root.m3u." Play that playlist to play all songs. If you are using the Tag Cache, then simply select album or artist and the "all files."
You can find more information in the Rockbox Manual. See the link in my signature.