buggy dircache activity renders ipod almost completely unresponsive

[squidkidd]

Funny, I had come to post that this problem had returned when I found several others had beat me to it.  Same deal for me, using the current SVN, playback resume, dircache enabled, database update disabled.  I first noticed it on Monday's build.

Regards,
Brian

[dip]

I have now checked build 17451 as mentioned by surfer and can confirm that with this build the problem mentioned in my post of May 24 is gone.

I further noticed that with the present build not only browsing the database makes the ipod unresponsive if at the same time dircache is initialized, but that playback is repeatedly interrupted over several minutes when playing music during dircache is initialized. This does also not happen with build 17451.

[jhMikeS]

I would like to know if you could let the disk spin down after boot (wait five or so seconds after that as well) instead of doing all the dircache/database/resume playback right at boot. After that, run the updates manually and start playback to imitate the startup situation as closely as possible and see if the symptoms show at that point as well.

[dip]

If I wait after reboot until the disk has stopped spinning (takes about 30-40 sec) I can browse the database and start playback without any problems, that means no lag during browsing and no interruptions during playback. Initializing dircache is shown in dircache info to have taken always about 30 seconds in this case.

In contrast, when I start browsing or playback before disk has stopped spinning, time for initializing dircache is much longer, e.g. indicated between 90 and 300 seconds. Sometimes it is then even indicated that dircache is not initialized.

[surfer]

http://In contrast, when I start browsing or playback before disk has stopped spinning, time for initializing dircache is much longer, e.g. indicated between 90 and 300 seconds

Yes, i can confirm the same: when i start a song during the initial scan (without 'resume playback' set as start screen) all the symptoms as with 'resume playback' are there again.

[Llorean]

What he wants you to do is turn *off* automatic database updates. Then wait for the disk to spin down after booting, start a database update, and then resume playback, to see if you have similar problems then.

Basically, the idea is to see if it's the updates that are causing the problem, or if it's relating to when the updates happen, rather than just what they do. At least, testing this theory as best as reasonably possible.

[jhMikeS]

Thanks Llorean. More specifically, this allows the HD power to be cycled once and see if things change. The HD power won't be turned off until several seconds after the HD has spun down from no activity. Then we want to have similar activity to the initial boot situation after repowering it and see if unresponsive UI or skipping is there.

I'll state again that it is by design that database update would take longer during playback than not during playback because it is a background update after all and takes a back seat to decoding audio. It will be true regardless of revision used.

The unresponsiveness and skipping that happens without the ata hack is utterly peculiar to the 60/80GB Video.

[surfer]

Please be aware as well that at least me and the starter of this thread, squidkidd, have this issue independently of the database.

I do not use the db and have never initialized it. It seems to be something related to the scanning for the dircache.

Some changes seem to have been made related to the dircache on one of the builds 17463-9 (2008-05-12) which introduced the problem (again).

[dip]

I have now turned off automatic database updates, rebooted and waited until the disk stopped spinning (which took again about 30 sec. as with automatic database updates on). Then I selected update database now and started immediately browsing (the database not the file tree). In this case there was no lag in browsing although database was updated in the background.

Since I had the impression that the problem could be related to the loading of dircache into RAM I in addition turned off this feature. Only for completeness in a first test I waited after reboot until the disk stopped spinning which took again about 30 seconds. As expected, browsing after disk stopped spinning was slower than before and you could see the "searching... " popup which counts the found albums probably since database had now to be read from disk and not from RAM.

In a second test after a further reboot I started browsing during the disk was spinning. In this case browsing took again much longer and the "searching..." popup counter did not continuously increase but was regularely interrupted. But I had the impression that the ipod was still a little bit faster than when automatic database update and loading dircache to RAM was on.

Then I turned loading dircache to RAM on again and after a reboot I started browsing the database during the disk was spinning. Then I had almost the same bad unresponsivness as at the very beginning.

So it seem to me that there is some disk activity after a reboot which take about 30seconds (after the Rockbox logo dissapears) and which catches almost all disk ressources. This seems not to be the automatic databse update and also not the dircache loading to RAM although the unresponsiveness seem to be even worse when both features are additionally turned on. In the latter case probably browsing has to fight not only with the unknown disk activity process but in addtion with the update process and the loading to RAM process.

Does that make sense?

[Sherv]

Adding my voice to the chorus. Running r17659-080530 and I experience the same problem if I resume playback immediately after booting. Waiting for the disk activity to cease before doing anything else (directory cache is on, database loading to RAM is off, last.fm logging is on [I read at one point that this may be a culprit]) results in clean and responsive playback.

[jhMikeS]

Nothing can catch all disk resources. The various threads trying to access the disk at the same time are granted it in first-in-first-out order so anything waiting for it will be given access in turn. The only major disk activity at boot would be the dircache and database scans. If both dircache and database auto update are both off and you're getting 30s of disk activity at boot then that's not right at all. It should stop after the disk spindown timeout which defaults to 5 seconds.

Clearly some things will take longer if the various caching schemes are disabled; that's why they're there. I of course experience that on any of my players as well-- including flash players-- and know it's normal. However, the sorts of changes within the daily build mentioned previously have no negative impact on any of them (look below this post for a list which actually includes an occasionally borrowed 5.5g 30GB).

What's interesting so far is that you don't have a problem with database updating in the backgound after HD has had a chance to cycle off. Now, check with dircache disabled, database auto update on and don't wait for the HD to stop at boot. Compare with the first two tests which sounded normal.

[dip]

I have now disabled loading dircache to RAM, set database auto update to on and then started browsing the database without waiting that the HD stopped spinning. The result was the same as when I disabled dircache AND automatic database update (that means it takes ages until the menu entries are shown and the "searching..." popup counter is repeadeatly interrupted). This was expected since why should setting automatic database update on improve the situation.

But then I found (hopefully) the reason for the intial disk spinning. I usually have set the option to show the database after startup. I now have changed that to show the main menu and after the next reboot the initial 30 sec disk spinning was gone. It seems that showing the top level of the database menu after reboot causes the disk spinning which then conflicts with other disk access like browsing the database or playback. Could the be the reason for the problem?

[squidkidd]

I'm using resume playback on startup and I hardly ever use the database, so there is still something in there besides the database access that is spinning the disk and making playback skip.

Regards,
Brian

[dip]

But then I found (hopefully) the reason for the intial disk spinning. I usually have set the option to show the database after startup. I now have changed that to show the main menu and after the next reboot the initial 30 sec disk spinning was gone. It seems that showing the top level of the database menu after reboot causes the disk spinning which then conflicts with other disk access like browsing the database or playback. Could the be the reason for the problem?

Sorry, I was wrong, the problem is independet from the chosen start menu. I double checked and could not reproduce that hd spinning does not occur when start menu is the main menu. So my conclusion was wrong, sorry for that.

There is alwas about 25 sec hd spinning after the rockbox logo disappears when dircache and database update is off. When dircache is on spinning takes about 5 sec longer. And when automatic databse update is in addition on there are another 5 seconds. But the initial 25 seconds are always there and cause the problem

[jhMikeS]

So, the virtual LED is on during this period indicating actual activity? This may be irrelevant depending on the following: be absolutely sure it's not just the HD spindown time set to 25 sec as well under settings|general settings|system|disk|disk spindown.