Rockbox Technical Forums

Installation / Removal => Rockbox Utility => Topic started by: marksz on September 01, 2009, 11:18:42 PM

Title: Cache Obfuscation and Offline mode?
Post by: marksz on September 01, 2009, 11:18:42 PM
The offline mode checkbox under the Cache tab is in a fairly obscure place to be really useful.
It's hard to remember that box is even there in an application that is used only occasionally.

I think it would be helpful at the very least if when running an install where there is no internet connection, the error message would at least hint about using offline mode in the cache configuration instead of just dying with "Download error: 2".  The first time I encountered this I assumed the program was designed to only work with an online index to updates and the cache.  I was wrong, but unable to access Rockbox on my player until I got back home where I have Internet, and then later noticed the checkbox for offline mode.  At best, when unable to contact the Rockbox web site for updates, it would be nice if the UI would gracefully ask the user if they would like to switch to offline mode rather than just die (or maybe also just if there is something in the cache to start with).

I'm running Rockbox Utility 1.2.2 (Rev. 21319)

I'm also irritated with the obfuscation of the files in the cache.  Right now my cache folder looks like this:


08/29/2009  12:41 PM         2,920,623 18262e1fe62650f55d79dcc69099d51c
08/29/2009  12:41 PM         3,385,059 59fa9b87a392b1786148570e4e8d0aa8
09/01/2009  10:41 PM               438 72fc9c40004cc4be5a6e56bf804e195a
09/01/2009  10:41 PM                56 976072f4dfe0affd68efba0b21112d1a
08/29/2009  12:40 PM            74,752 9ab75f4f72a787c16b50987823eaaafe
08/29/2009  12:42 PM         6,914,626 9e1166c96aee68ce7c73558bf6778194
               6 File(s)     13,295,554 bytes


I can guess from the file size that the 2nd file is rockbox-fonts.zip which I've already downloaded several times, but why not leave files in the cache with their original filenames?
If I want a permanent archive of a particular file the utility has already downloaded, I'm going to download it again wasting bandwidth rather than use the copy I already have as these obfuscated file names make it too hard to figure out what's what.  It also wastes my drive space as I'm storing two copies of the same file with different names.

If there is meta-data encoded in these file names, why not append the real filename at the end like 59fa9b87a392b1786148570e4e8d0aa8-rockbox-fonts.zip instead of just having hex code for filenames?   Or better yet, make the meta data all or mostly human readable!


Thanks to anyone who's willing to explain why it is this way, and even more thanks to anyone inspired to update the code to make it more user friendly along the lines I've suggested!

I'm very new to Rockbox, so I hope I'm not missing something obvious to those more familiar with it.
Title: Re: Cache Obfuscation and Offline mode?
Post by: Llorean on September 02, 2009, 12:10:01 AM
The "obvious" thing you're missing is that Rockbox Utility really isn't meant for offline operation. You need to be online to acquire Rockbox utility and an up-to-date build.

As well, the cache isn't really there for the files to be human readable - if you're using the utility they don't need to be and if you're not using the utility then why do you need it to download and cache files in the first place?
Title: Re: Cache Obfuscation and Offline mode?
Post by: marksz on September 02, 2009, 01:24:29 AM
If I'm working with several players of the same model, why do I need to stay online for them all?  It seems I don't need to, now that I know where the configuration option is for offline. 
If it's not meant to be used offline, why does it work so well offline once one knows to check the right setting?  It certainly is useful to me to be able to run it offline.  If I choose to use a major release rather than a daily build, or to stick with one build for a while on several players, offline mode is very useful, esp. when I'm traveling (and that is when I use Rockbox the most for audiobooks on road trips).

I bought several e200 series Sansa players for my friends, and installed Rockbox for them.  One person found Rockbox too confusing, and had trouble remembering which key to hold to revert to the Sansa firmware while booting and asked me to "fix it so Rockbox would not load".  Then minutes later asked me to change it back.  We didn't have Internet where we were meeting over lunch, but I knew I had all the necessary files on my computer.  I just didn't remember that offline check box was there to make it work, so my friend is out his player until I see him again (which may be some time) as I brought it home to reinstall the boot loader online as he doesn't have Internet at his home.  It turns out that I could have reinstalled the boot loader immediately, it's just that the software isn't currently user friendly enough to help me do that quickly.

My main point is that it would be more user friendly to mention this option or offer to enable it when needed, and the program certainly could look to see if this might be useful (i.e. if files exist in the cache).  Why does it default to give up even though it already has the necessary file but just doesn't bother to look unless I know to check an obscure option?  I think software should be designed to make accomplishing goals easy, and not require me to keep track of obscure settings, or at least have a helpful error message that could jog my memory!

An example of when I might want to use the cache file would be to copy the same release, or an individual file from one, to the simulator on my PC where I'm testing a new WPS or something, or if I want to compare files in one release to those in a different build.  I also have purchased several similar players, and would like to be able to install the same Rockbox I have already on one to another spare even when I can't get online.  It would be less confusing to be sure I have the same build or release on each.

There are probably several more examples of different ways to use at least some of the files that get stored in the cache.  I find it wasteful that the obfuscation encourages unnecessary duplication of storage, bandwidth use, and my time to find and download files I already have.

I'd like it if there were some way I could get the utility to store my preferred settings, WPS or Theme etc. so if I can easily set up a duplicate player the same as one I've been tweaking for a while.

I also think it's usually helpful to developers to hear from users who use their software in ways that they did not anticipate in the early design of an application.   Usually the idea is to make the software as useful as possible, so easily implemented options for different uses add to the overall benefit of the application.
Title: Re: Cache Obfuscation and Offline mode?
Post by: Llorean on September 02, 2009, 01:48:26 AM
You can't copy the same release to the simulator - simulators *are* the Rockbox build. It must use 100% simulator files. So, this example is invalid.

As to the prior ones - yes it could help if the option were made more noticeable, but if you're installing the same copy on multiple devices you still don't need to directly access the cached files.

As for "easily setting up a duplicate player as one I've set up before" - read the installation instructions in the manual. Once you've installed the bootloader you can literally simply copy the .rockbox folder to the new player to be duplicated (assuming they're the same hardware) and you're done. No need to try to do complicated things with the utility.

Sure, sometimes users want to do things the software wasn't intended for. But often, the software wasn't intended for it because it simply doesn't make sense to have it do a job as simple as copying a single folder.
Title: Re: Cache Obfuscation and Offline mode?
Post by: bluebrother on September 02, 2009, 01:57:39 AM
The offline mode checkbox under the Cache tab is in a fairly obscure place to be really useful.
It's hard to remember that box is even there in an application that is used only occasionally.

I don't think that's obscure at all. Offline mode means everything has to be (and gets used from) the cache, so where should it placed instead?

Quote
I think it would be helpful at the very least if when running an install where there is no internet connection, the error message would at least hint about using offline mode in the cache configuration instead of just dying with "Download error: 2".

Offline mode doesn't work perfectly and is not announced anywhere. This is intended, at least until Offline mode works better -- or gets removed, as it does have issues. Most parts of Rockbox Utility require a network connection and haven't been written with offline mode in mind.

Quote
it would be nice if the UI would gracefully ask the user if they would like to switch to offline mode rather than just die (or maybe also just if there is something in the cache to start with).

There's the problem: Rockbox Utility is intended to provide information about the latest version available. Without network connection this is not possible, and for that reason the information about the latest version is (intentionally) not cached. Offline mode enables this. Always caching those information (you usually only need once) brings the possibility for errors of not noticing when the file on the server changed.

However, this is also a leftover from the times when the caching mechanism wasn't as smart as it is these days. When caching was introduced a cached file was always used in preference of a file on the server, regardless if the file on the server was newer than the cached one. It works differently these days.

Quote
I can guess from the file size that the 2nd file is rockbox-fonts.zip which I've already downloaded several times, but why not leave files in the cache with their original filenames?

Because directly after download the files don't have a filename (they are downloaded to a temporary location, and the filename for that temporary file is automatically generated). More important, there are several files on the download server that can only get distinguished by the path but not the filename (until recently the new build system was implemented all current builds had the filename "rockbox.zip", so there was no way to distinghish anything from the filename!). Thus, caching md5sums the complete URL, which means that we can reliably find out if the file in the cache is actually the file to be retrieved -- or a different file with the same filename. Also, if retrieving data from locations that need additional GET parameters the plain filename won't work either.

Quote
If I want a permanent archive of a particular file the utility has already downloaded, I'm going to download it again wasting bandwidth rather than use the copy I already have as these obfuscated file names make it too hard to figure out what's what.  It also wastes my drive space as I'm storing two copies of the same file with different names.

That's true, but for one disk space is cheap these days. Second, the downloads aren't that big. And last, how often do you want to create a local archive of a file? It's really just a rare case that users want to do this, and using hashes makes the cache much more robust.

Quote
If there is meta-data encoded in these file names, why not append the real filename at the end like 59fa9b87a392b1786148570e4e8d0aa8-rockbox-fonts.zip instead of just having hex code for filenames?   Or better yet, make the meta data all or mostly human readable!

Because it is additional work without any real benefit. The only thing we might consider is to write a file alongside the cache holding the URLs present in the cache with their md5sums.

Quote
Thanks to anyone who's willing to explain why it is this way, and even more thanks to anyone inspired to update the code to make it more user friendly along the lines I've suggested!

Well, as caching is nothing a user should have to care about or fiddle around with manually (how often have you browsed the cache files of your browser?) I'm not willing to change anything here -- except considering the additional "translation" file I mentioned. Nevertheless, there are quite a bunch of issues in current Rockbox Utility that have way higher priority -- it's just caching, and it works fine. Feel free to send patches :)