iPod Video 5G with iFlash adapter - is RockBox really hopeless in that case?

[RockBoxNewB]

By the way, I forgot to ask again: for the time being, RockBox seems to be running better than ever, particularly as far as its general response time goes. But once again, there's one thing I am still mourning about: LOAD TO RAM. My personal habits involve a heavy use of the database feature, and whenever RB has to go back to a list of songs within the database, it takes a long time to load (even smaller lists). The battery drain is also heavy because of this. I could imagine that just like on my other, "still-HDD-based" iPod, load to RAM could help excellently with both these issues.
Sorry to annoy you with this, but: Is there any way somebody could supply a pre-filesystem-rework build with LOAD TO RAM still included?

Thanks in advance,
RBNB

[cereal_killer]

By the way: which files do I need to copy between devices in order to import the database? Does the "changelog" suffice or do I also need one of the other files labeled "database..."?

I don't know if it is possible to transfer the database from one device to an other, but the files labelled database_*.tcd contain the database. As long as the file structure is the same on both devices, copying those files could import the database.

Is there any way somebody could supply a pre-filesystem-rework build with LOAD TO RAM still included?

If you use such a build, you will not have all the new features and bugfixes that were committed since then.

Better than waiting for someone to supply such a build is making your own. I recently made myself familiar with compiling and stuff, it is sometimes not easy, but so great if you manage to compile your own build. This is also more in line with "hacking" and "modding" like you do it with your iPod. So I suggest to have a look at http://www.rockbox.org/wiki/DevelopmentGuide and http://www.rockbox.org/wiki/LinuxSimpleGuideToCompiling

If you have Problems, there are many helpful people here who you can ask.

I hope you achieve what you want and enjoy your rockboxed and modded iPod.

Edit: Didn't [Saint] provide you with such a build? (http://forums.rockbox.org/index.php/topic,51191.msg236396.html#msg236396)

[RockBoxNewB]

Hi,

Starting with your last point first: [Saint] provided me with a build compiled to include load to RAM - but that one was for my other iPod, a 6G 160 GB model. Not knowing better, I even tried that build in the first place - but it lead to an error on my 5G model which then refused to start RockBox after all. I assume both models are completely incompatible in that respect...

I have copied over the RockBox database between devices several times (or let's say: I attempted to do so). Sometimes it worked, sometimes it didn't. I think migrating it between different manufacturers is something I haven't managed to do so far (I think this concerned both moving from a Sansa Clip Plus to a Cowon i5 as well as moving from the Cowon to the iPod). Moving between iPods worked so far, except for some hiccups and partial loss of database information.

I took a look at some links on compiling. Although I might know my way around stuff better than I did some years ago, I'm afraid that skill-wise, I'm VERY far from successfully compiling my own build. I assume I'd rather brick my iPod a dozen times than actually being able to make it work, I guess!

Best,
RBNB

[[Saint]]

By the way: which files do I need to copy between devices in order to import the database? Does the "changelog" suffice or do I also need one of the other files labeled "database..."?

I don't know if it is possible to transfer the database from one device to an other, but the files labelled database_*.tcd contain the database. As long as the file structure is the same on both devices, copying those files could import the database.

Don't do this.

The Database context menu has export and import options for precisely this purpose, this exports to an ASCII plain text file that guarantees forward and backward compatibility.

Is there any way somebody could supply a pre-filesystem-rework build with LOAD TO RAM still included?

If you use such a build, you will not have all the new features and bugfixes that were committed since then.

Better than waiting for someone to supply such a build is making your own. I recently made myself familiar with compiling and stuff, it is sometimes not easy, but so great if you manage to compile your own build. This is also more in line with "hacking" and "modding" like you do it with your iPod. So I suggest to have a look at http://www.rockbox.org/wiki/DevelopmentGuide and http://www.rockbox.org/wiki/LinuxSimpleGuideToCompiling

I'm going to go ahead and guess you're not familiar with the massive complexity surrounding the filesystem rework, and that's fine. I don't expect you to.

If this was a simple change I would have done it a couple of years ago. If this user wanted to retain the current feature set and bugfixes, you're looking at multiple tens of man hours work for an experienced embedded developer, likely hundreds for an inexperienced one, and borderline impossible for Joe Everyday,

This is significantly more involved that just tracking down a single (or even a couple of) commit(s) and reverting it(|them). The idea of cleanly reverting the current code base to the legacy filesystem code and keeping the feature set and bugfixes added since then is basically an idea that isn't worth entertaining. The change touched hundreds of files in thousands of locations, and because of its very low level interaction, pretty much every non-trivial change or addition since then is directly or indirectly tied to these changes.

Not least of all is that this is only really interesting for testing purposes, we're only interested in if performance has indeed degraded, and if so by how much. We don't need to bring a legacy function up to the current code base in order to test this.

If it turns out that there is a significant performance hit, there's basically two options going forward:

 - completely revert the filesystem rework and then rebase ~75% of all the commits added since (....Not gonna happen. Period. You can take that to the bank).
 - re-implement an analog to the 'cache in RAM' feature against the modern filesystem and core (Waaaaaaaaaaaaaaaaaaaayyyyyyyy less work, but still non-trivial, and not an area of the code base that I am at all comfortable or familiar with. The Database function and its interaction with the filesystem has remained largely untouched for years because it is made of prayers, good intentions, bailing twine, and magic).


[Saint]

[RockBoxNewB]

Hi [Saint],

Thanks for your once again "enlightening" insights.

By the way: which files do I need to copy between devices in order to import the database? Does the "changelog" suffice or do I also need one of the other files labeled "database..."?

I don't know if it is possible to transfer the database from one device to an other, but the files labelled database_*.tcd contain the database. As long as the file structure is the same on both devices, copying those files could import the database.

Don't do this.

The Database context menu has export and import options for precisely this purpose, this exports to an ASCII plain text file that guarantees forward and backward compatibility.

What I was asking (more or less) is: is the database_changelog the ASCII plain text file you were talking about? Because using the import/export options from the menu is exactly what I did, I was just wondering whether it is just the changelog file which I need to copy to the other device for import?! When I did that, it worked after all, but with the hiccups I mentioned, more precisely some dozens of files not receiving the info which was actually already stored in the database.


To be honest, what you say about compiling is pretty much what I expected. It would have surprised me A LOT if I was able to get up to speed quickly to indeed create my own build for the iPod. This makes me even more overwhelmed about how you managed to compile the build for my other iPod 6G that included Load-to-RAM. I assume this is not possible for the 5G Video?
As an alternative (although, I know, you don't recommend it), is it possible to get just the old build with Load-to-RAM, without the bug fixes made since? Without anyone having to compile or modify anything, just the plain build that was available before the filesystem rework? I searched around a bit, but I could only find builds that are max. a couple of months old.

Best,
RBNB

[cereal_killer]

Is there any way somebody could supply a pre-filesystem-rework build with LOAD TO RAM still included?

If you use such a build, you will not have all the new features and bugfixes that were committed since then.

Better than waiting for someone to supply such a build is making your own. I recently made myself familiar with compiling and stuff, it is sometimes not easy, but so great if you manage to compile your own build. This is also more in line with "hacking" and "modding" like you do it with your iPod. So I suggest to have a look at http://www.rockbox.org/wiki/DevelopmentGuide and http://www.rockbox.org/wiki/LinuxSimpleGuideToCompiling

I'm going to go ahead and guess you're not familiar with the massive complexity surrounding the filesystem rework, and that's fine. I don't expect you to.

You are right, I can not imagine what it means to develop or write code for such a complex thing as the filesystem, I didn't suggest to rewrite the code, but I tried to encourage the OP to learn how to compile a pre-filesystem-rework build on his own. Thanks to the great documentation, this should be feasible for an advanced computer user within a few days.

[RockBoxNewB]

Alright guys, considering that my computer knowledge won't be "advanced enough" to compile a build myself, I tried another way to solve the problem - or rather: go around it. the 5G iPod Video with extended flash storage was actually only planned as a "pilot project" of sorts. I wanted to test the procedure before I modify my 6G iPod Classic 80GB (and possibly brick it in the process). I decided to do this know because I felt confident enough not to damage the device, and because I considered it interesting to see how RockBox runs on a device with more RAM and a better processor.

The installation worked without considerable problems. Unfortunately, the 6G iPods have a 128GB limitation within iTunes, so it would not show the full 384GB of storage (this time I used the three 128GB micro SDs again). It is possible to "unlock" the full capacity using Rockbox - but this also means that it is no more possible to use the "safe file transfer" procedure by booting into the Apple OF before connecting USB. Anyway, I was able to transfer all files without the connection crashing and as far as I could judge from a short test, there seemed to be no skips during playback or anything.

But then - the database again. As opposed to the 5G iPod Video, until now, I have NOT been able to successfully transfer the database to this iPod. This would be a major dealbreaker for me. The following happens, and I have never had that so far with all the different database transfers between devices which I have done so far: After transferring the database_changelog.txt from the old device over to the 6G iPod, I hit "Import modifications" from the database menu. As usual, the screen freezes, displaying "Loading". This time, however, it would never awake again. For some minutes, it is possible to turn screen backlight on by pushing any button, where you can still see that it is "loading". After some minutes, however, the iPod would stop reacting to anything. Backlight cannot be reactivated, and it also cannot be turned on by pushing "Play". This could suggest that the device is somehow still turned on. The only way to get it back to life is to do a hard reset by holding "Menu" and "select". I have given it around 30 minutes time so far, the situation would not change. I will now leave it lying aside for a couple of hours, just to make sure that it is just a matter of time until it miraculously wakes up again.

Does ANYBODY have any idea what might be going wrong here, anything? PLEASE, I would be so grateful for any support. There is only one small step left to do to achieve what I was trying to do, and it would be a huge setback if the entire project would fail because of it.

Many thanks,
RBNB

[RockBoxNewB]

As an addition to my last post, this might help with the diagnostics: After attempting the database import procedure again and again, I got it to finish 2 or 3 times. However, a significant part of the database was always missing. After trying around a bit, I found out that the result would always be identical: precisely 2471 out of around 7500 files would remain without their proper database entry (in my case, I could judge that from the missing RATING which I gave to the tracks).
This sounds like one particular entry/file/whatever causes an error and therefore the procedure to abort, or what do you think? Is there any way to find what causes the error?

Thanks,
RBNB

EDIT: It occurred to me that I read about the "metadata log" somewhere else on the forum, which might give a strong indication if the file parser fails with always the same audio file. I activated it from the System > Debug menu, but... how can I READ that log now? The RB documentation says something about .metadata/log, but I can't find that folder anywhere on my iPod.

[RockBoxNewB]

Hi everybody, particularly [Saint] and all those who might have followed the discussion earlier on the IRC channel.
I've got some new insight that might help finding the cause of the problem. I strongly assume that the file corruption (if there was any) must have occurred at an earlier stage, with the database import procedure actually doing its job perfectly (the metadata log showed that all files were processed, with no crash happening at any point).
This is what I found out: While I was testing the device (an iPod 6G 80GB modded with Tarkan's iFlash Quad adapter to hold three 128GB micro SD cards), I noticed stark volume differences between tracks. Usually this doesn't happen, since I use ReplayGain on ALL of my tracks. I checked it and found out: all tracks which seemed to have lost their metadata, also had lost their ReplayGain information! To me, this looks like the files were corrupted already when I transferred them to the device via USB, with the database function obviously not "recognizing" them as the files that the changelog.txt provides metadata for.
Since the 6G iPods have a 128GB limitation when formatting via iTunes, it's not possible to go the "safe way" by starting the Apple OF and transferring the files from there. It HAS to be done via Rockbox (there are often problems with RB's USB handler causing file corruption). Tarkan himself told me that since the new iPod 6G bootloader is being used and EMCORE is obsolete, the USB handler problems were supposed to be corrected - but I'm not sure this is entirely true.
Please let me know if this helps.

Best,
RBNB