pictureflow problem - any suggestions for a fix/workaround?

[Frankenpod]

I can't build the database for pictureflow, because it goes for ages then fails with an 'insufficient memory' error.  The error is more specific than that, it refers to a particular kind of memory, but without spending ages letting it run again I can't remember what the exact error message was, sorry.

I presume it's because this is a flash modded ipod and has a lot of tracks (> 10,000) so a lot of cover art.

  Alternatively, possibly it's to do with the usual problem with writing to the drive on flash-modded ipods...but my impression is that has gotten a lot better with later dev versions (this was using a very recent dev version, i.e. within the last couple of weeks...it seems more recent versions are more stable than RB used to be when it comes to building the database on modded ipods, for example, so I'm wondering if now the problem with pictureflow is simply what it says it is - too much cover art for the allocated memory?).

Does anyone have any ideas as to how to work around this?

I might try a test with an iFlash ipod with not many tracks on it, to check if it really is just a matter of too much cover art for the plug-in to cope with.

[Bilgus]

Pictureflow uses the remaining plugin buffer to create the list of tracks and then writes it to a database file
I recently rewrote it to do t :(his in in stages likely it's just out of space..
We could probably stop playback and take the audio buffer too if it's not already doing that
During building

I don't remember if the album art is resized before pf copies it but if so that could be an issue

[Frankenpod]

I notice there have been some fixes to pictureflow by Chris_s recently.  Would those relate to the kind of crash I was getting?  i.e. is it worth trying again with a recent dev version of rockbox?  Or were those fixing some other, unrelated, bug/crash issue?

On a related note, as there seems to have been a lot of changes (some reverted?) to both the database handling and pictureflow in recent dev versions, I don't suppose there's any chance anyone would do an update of the Windows-application simulator builds (as downloadable under 'extras')?

 Perhaps that's asking a lot as I guess it would be a lot of work to build those for every player.  But the changes to the database code means that one can't now use the simulators (as currently available) to build the database on the PC and copy it to the device.

(I notice at least at one point, the database lost the separate artist/album artist tag choices...curious what the reasoning for that was.  It's not a big deal, though usually I'd prefer to use the 'album artist' tag, as it avoided including entries for large numbers of ultra-obscure artists that I just happened to have one lone track from on a compilation album - making it easier to find the artist I was actually looking for...and conversely, sometimes I'd use the 'artist' field, if I was looking for one of those obscure compilation-album-only artist tracks.)

[chris_s]

I don't think my PF fixes will be helpful with respect to the crashes you were getting building the album index. Have you tried using smaller artwork files as Bilgus, I think, alluded to? I have about 17k songs and neither the iPod 4g nor iPod video (both modded using an "iFlash-Solo" adapter)  seem to have trouble building the index on device (although it takes a long time and I have to make sure the device doesn't go to sleep in between) with 160x160 JPEGs.

As for the album artist entry, I'm with you, it's also been discussed on IRC and Gerrit. Like you said, PictureFlow at least has been reverted to using album artists again. Which may mean, although I could very well be wrong, that the old Windows Simulator applications can once again be used to build a PF database for the latest dev builds... I don't know who'd be able to update them otherwise..

[Frankenpod]

Thanks (and thanks for your ongoing work on Rockbox).

I guess I could try shrinking the artwork files - would have to set up some series of scripts to extract all the embedded art, then resize it, then re-embed it, much as I did when purging all the .png format art, but it would be a bit of effort and haven't got round to trying it yet.  Presently most of the art is about 300x300 pixels, not sure how much shrinking would be needed to get picture flow DB to build - and I'm not certain a memory-shortage issue is the only problem there anyway.

  When it gives an error message (right at the very end after taking an absolute age to go through the stages of building the pictureflow DB) about 'insufficient memory', is it referring to onboard RAM, or does it mean it's having trouble writing stuff to the flash card?

[chris_s]

For what it’s worth, the PF plugin, unlike the playback engine of Rockbox, doesn’t look at embedded artwork, only at separately stored artwork files, so you should probably make sure the latter are of an appropriate size. The order used by Rockbox is also described in the manual: https://download.rockbox.org/daily/manual/rockbox-ipod6g/rockbox-buildap3.html#x19-430000C

My understanding would be that “insufficient memory” refers to the plugin buffer only (so part of RAM), but Bilgus is obviously better equipped to speak on that.

[Frankenpod]

For what it’s worth, the PF plugin, unlike the playback engine of Rockbox, doesn’t look at embedded artwork, only at separately stored artwork files, so you should probably make sure the latter are of an appropriate size.

That's helpful to know.  At some point I guess I'll try bulk-resizing all the cover.jpg files (leaving the embedded art untouched - I mostly have cover.jpg files in each directory with just the first track having the same art embedded in it), and see if that makes any difference with building the PF DB.

[Frankenpod]

Drat.  Even after downsizing every cover.jpg to 160 pixels width, I still get the 'insufficient memory for album art cache' error message at the end of the initialising process.  Not sure what is the issue.

[amachronic]

"Not enough memory for album art cache" means the artist & album index is too big and there is not enough memory left over for buffering the cover images as they are loaded. You can look for the pictureflow_album.idx file (it'll be somewhere under the .rockbox folder) and see how big it is. The size of that file should be approximately how much RAM is being used for the index. If it's too big, that'd be the problem.

The size of your cover art shouldn't matter too much -- I think it's only read in small chunks and gets re-scaled on the fly.

[Frankenpod]

Thanks.  I'll have a look and see if it's there and what size it is - assuming it actually succeeded in creating that file - it seemed as if the whole initialisation process failed at that point, but not really sure.

[chris_s]

I've noticed the plugin buffer size is only set to 800KiB on all iPods including the iPod classic, despite it having 64MB of RAM, whereas, for example, amachronic seems to have set it to 2 MiB on the M3K with the same amount of RAM. Might be worth trying to simply increase that at bit, at least, in a one-off build for yourself?

[chris_s]

Or 512KiB actually...

[Frankenpod]

Will try that - though, unfortunately, my linux PC (that I'd have to use to build RB with) is currently partially-dismantled!

[Frankenpod]

Gave it another go with the latest RB version.  Took 4 hours to complete then said 'not enough memory for art cache'!

the pictureflow_album.idx file is 304k, which doesn't seem that large.

[chris_s]

Gave it another go with the latest RB version.  Took 4 hours to complete then said 'not enough memory for art cache'!

the pictureflow_album.idx file is 304k, which doesn't seem that large.

Doesn't really solve your problem, but building the index shouldn't take that long. Have you enabled 'Load to RAM for the database (and restarted afterwards)? I have just under 20k songs on my iPod 4G, resulting in an album index that's about 130 KB, and it takes  (I just timed it) under 2 minutes to build the index and reach the "Preparing artwork" stage.

You're using an iPod 6G, if I remember correctly? Here's a dev build with the plugin buffer size set to 2MiB instead of 512KiB, if you'd like to give it a go: https://www.dropbox.com/s/tyi960dv0halide/rockbox-full-ipodclassic.zip?dl=0