In to the /.rockbox directory.
Files such as the .wps, .bmps, etc. should all go in to their appropriate directories that way.
If you're unsure, extract it somewhere else on your computer and have a quick look to see what directories and files are produced.
But, yes, extracting it to the ./rockbox directory is what you ultimately want to do.