Save downscaled album art in database as cache

[Yakov5776]

Hey, the reason for this feature request is because every song I load takes a long time to load (5s-30s)
My proposal is to store a compressed version of the album art (e.g. 200x200) which is to be saved in the database so music with larger album arts won't take time to load anymore.

As of now, the only workarounds are either: (a) disabling album art covers or (b) downscaling/compressing album art manually using tools like mp3tag which is pretty tedious and requires A PC and modifying the original music file.

[saratoga]

How big is your album art? Generally resizing is relatively quick.

[Yakov5776]

My album sizes are 2000x2000, all my music use those dimensions.

Also, I'm not implying necessarily that the resizing aspect of it is what takes a long time (It could be loading the art into stream, parsing it, etc), All I'm saying is having this pre-resized in the database will guarantee performance for music with larger album arts that are anyways way above the resolution of the iPod screen.

[saratoga]

If the album art is embedded you have to parse it anyway to get to the audio, so only savings would be the resizing. At 4MP should be pretty quick I think.

Is it slow with embedded or folder jpg?

[Yakov5776]

Embedded

[saratoga]

I tried resizing a 2000x1500 folder.jpg on an e200v1 (slowest device I have) and it delayed about a second, maybe a bit less on track change.

I wonder if your files are slow if they're not embedded into files?

[Yakov5776]

Strange, this slowness does not happen to me when disabling album art.

Tomorrow I'll message you a zip of a few songs to try so you could try to reproduce the issue.
Also, they're in ALAC format in case that plays any role.

[saratoga]

The slower images are really big, 3000x3000 and several MBs which can take 5-10s in my testing.  Keeping the high resolution but compressing them a little more so that the files aren't so big largely resolves the slowness on my e200v1, so the issue is probably the time it takes to load and decompress the huge file more so than resizing.  Caching might help after all, but I don't know how easy that would be to implement.  Maybe a better option would be to skip album art above a certain size on slower devices.

[Yakov5776]

I had the chance to try it out on the latest rockbox windows simulators, and even on that it suffers a half-second delay, and while that may seem pretty negligible, it's no surprise that it takes a longer on more lower-end hardware, for example the iPod classic 5g which was released 2 decades ago (~5s delay). I had to disable the art album feature in favor of performance which is all the more reason I think something should be done even from an optimization perspective. I love this project and I love what you devs do to make this even possible in the first place, and I'm also grateful to anyone who has taken the time to look into this issue.

[Yakov5776]

For those reading this thread who may not be aware of the audio files I used and didn't receive the sample audio files, the album arts of my music files i have are typically ranging from 2k to 5k resolution (which is because they're retained in it's original quality)

Maybe a better option would be to skip album art above a certain size on slower devices.

while that would definitely be better than nothing, these devices are capable of processing these larger album arts if they were just tailored for their specifications, even if that entails storing a pre-compressed cached version at db building time, similar to like how iTunes compresses the album arts when syncing it to the iPod which is why iPodOS seems so swift and fast even while having a limited cpu perf.

[saratoga]

Apple converts the image files on the PC before copying them to the device, which is what's recommended when using rockbox too. If you want to look into doing it on device I suppose you could save an additional .bmp of the resized image and then skip the jpeg version on subsequent loads. Since you can't save as jpeg it's a little less efficient but the resized images would be small anyway.

[Yakov5776]

I suppose you could save an additional .bmp of the resized image and then skip the jpeg version on subsequent loads. Since you can't save as jpeg it's a little less efficient but the resized images would be small anyway.

Yeah, I started messing with the source to implement this functionality but it isn't very straightforward. buffering.c:load_image is where the AA image finally gets parsed and formatted in bmp format, but unfortunately, id3 data is already lost at that point and i don't have any other ideal way of caching the albumart bmp to a file name which can correspond with the album and artist name.

I already have logic to load that bmp cache filename if it exists before attempting to load the embedded art but I'm not so satisfied with the way it's being checked.

Code: [Select]

/* Check if a cached BMP file exists within the .rockbox directory
 *
 * id3      - metadata of the current track.
 * buf      - buffer to store the path of the cached BMP file.
 * buflen   - size of the buffer.
 *
 * Return true if a cached BMP file is found, false otherwise.
 */
bool check_cached_bmp(const struct mp3entry *id3, char *buf, int buflen)
{
    char path[MAX_PATH + 11]; /* need room for filename and null termination */
    const char *artist = id3->albumartist != NULL ? id3->albumartist : id3->artist;

    if (!id3 || !buf || !artist || !id3->album)
        return false;

    snprintf(path, sizeof(path), ROCKBOX_DIR "/albumart/cache/%s-%s.bmp",
             artist, id3->album);
    printf("Path: %s\n", path);

    if (file_exists(path))
    {
        strmemccpy(buf, path, buflen);
        logf("Cached album art found: %s", path);
        return true;
    }

    return false;
}

This is what I got so far and an integration with the database would be a lot better and possibly more efficient in terms of storing and indexing cached album art.
Any pointers would be appreciated. (Be easy on me, this is the first time I've worked on a vast open-source C codebase like this one)

Also on a side note: I was wondering why album art loading can't be done while the song is playing, why does it wait for that to load before it can actually start playing? It does load the default album art before loading the embedded one so you should assume that the embedded one would load on a separate thread from actual playback.

[saratoga]

Album art already has an ordered list of places to look on load. You just need to add one more location, wherever you cache the resized file. Then if the cached item is there, it will be loaded, if not it will continue down the list as normal.

The database is a system for generating playlists from file tags. Since album art doesn't really relate to playlists, it doesn't make sense to tie this to the database. Just make one cached item per file or per folder and you can use the existing load logic to pick the cached item when available.

[Yakov5776]

Album art already has an ordered list of places to look on load. You just need to add one more location, wherever you cache the resized file. Then if the cached item is there, it will be loaded, if not it will continue down the list as normal.

That's a good idea, although it should be an opt-in/opt-out situation so users have the choice to use album caches, as this may not be ideal for users who have a lot of albums and already have fast loading album arts and therefore the caches are redundant, I was thinking maybe there should be some dimensions qualifier before it gets stored.

The database is a system for generating playlists from file tags. Since album art doesn't really relate to playlists, it doesn't make sense to tie this to the database. Just make one cached item per file or per folder and you can use the existing load logic to pick the cached item when available.

That's true, but the database system was modulized in a way where you could add other processes to the procedure and it doesn't have to be strictly limited to just indexing file tags.
In fact, I think it's more befitting to cache album arts while the database is indexing so then you don't don't have to play your whole music library before you have subsequent caches for any large album arts.

[saratoga]

I think you should get your caching system working first and then think about how you want to rework other systems.