Problem filtering by year

[ilmioalias]

Hello,
i'm trying to filter songs by year subsequently by album; i created this custom tagnavi (only a snippet):

Code: [Select]

%menu_start "custom_artist_initial" "Browse album artists"
"Non-alphabetic"   -> albumartist ? albumartist < "A" & filename !~ "_cmpl" -> year -> album  ->  title = "f_browse"
"A"   -> albumartist ? albumartist ^ "A" | albumartist ^ "The A" & filename !~ "_cmpl" -> year -> album  ->  title = "f_browse"
"B"   -> albumartist ? albumartist ^ "B" | albumartist ^ "The B" & filename !~ "_cmpl" -> year -> album  ->  title = "f_browse"
"C"   -> albumartist ? albumartist ^ "C" | albumartist ^ "The C" & filename !~ "_cmpl" -> year -> album  ->  title = "f_browse"
"D"   -> albumartist ? albumartist ^ "D" | albumartist ^ "The D" & filename !~ "_cmpl" -> year -> album  ->  title = "f_browse"
"E"   -> albumartist ? albumartist ^ "E" | albumartist ^ "The E" & filename !~ "_cmpl" -> year -> album  ->  title = "f_browse"
"F"   -> albumartist ? albumartist ^ "F" | albumartist ^ "The F" & filename !~ "_cmpl" -> year -> album  ->  title = "f_browse"
"G"   -> albumartist ? albumartist ^ "G" | albumartist ^ "The G" & filename !~ "_cmpl" -> year -> album  ->  title = "f_browse"
"H"   -> albumartist ? albumartist ^ "H" | albumartist ^ "The H" & filename !~ "_cmpl" -> year -> album  ->  title = "f_browse"
"I"   -> albumartist ? albumartist ^ "I" | albumartist ^ "The I" & filename !~ "_cmpl" -> year -> album  ->  title = "f_browse"
"J"   -> albumartist ? albumartist ^ "J" | albumartist ^ "The J" & filename !~ "_cmpl" -> year -> album  ->  title = "f_browse"
"K"   -> albumartist ? albumartist ^ "K" | albumartist ^ "The K" & filename !~ "_cmpl" -> year -> album  ->  title = "f_browse"
"L"   -> albumartist ? albumartist ^ "L" | albumartist ^ "The L" & filename !~ "_cmpl" -> year -> album  ->  title = "f_browse"
"M"   -> albumartist ? albumartist ^ "M" | albumartist ^ "The M" & filename !~ "_cmpl" -> year -> album  ->  title = "f_browse"
"N"   -> albumartist ? albumartist ^ "N" | albumartist ^ "The N" & filename !~ "_cmpl" -> year -> album  ->  title = "f_browse"
"O"   -> albumartist ? albumartist ^ "O" | albumartist ^ "The O" & filename !~ "_cmpl" -> year -> album  ->  title = "f_browse"
"P"   -> albumartist ? albumartist ^ "P" | albumartist ^ "The P" & filename !~ "_cmpl" -> year -> album  ->  title = "f_browse"
"Q"   -> albumartist ? albumartist ^ "Q" | albumartist ^ "The Q" & filename !~ "_cmpl" -> year -> album  ->  title = "f_browse"
"R"   -> albumartist ? albumartist ^ "R" | albumartist ^ "The R" & filename !~ "_cmpl" -> year -> album  ->  title = "f_browse"
"S"   -> albumartist ? albumartist ^ "S" | albumartist ^ "The S" & filename !~ "_cmpl" -> year -> album  ->  title = "f_browse"
"T"   -> albumartist ? albumartist ^ "T" & albumartist !^ "The A" & albumartist !^ "The B" & albumartist !^ "The C" & albumartist !^ "The D" & albumartist !^ "The E" & albumartist !^ "The F" & albumartist !^ "The G" & albumartist !^ "The H" & albumartist !^ "The I" & albumartist !^ "The J" & albumartist !^ "The K" & albumartist !^ "The L" & albumartist !^ "The M" & albumartist !^ "The N" & albumartist !^ "The O" & albumartist !^ "The P" & albumartist !^ "The Q" & albumartist !^ "The R" & albumartist !^ "The S" & albumartist !^ "The U" & albumartist !^ "The V" & albumartist !^ "The W" & albumartist !^ "The X" & albumartist !^ "The Y" & albumartist !^ "The Z" & filename !~ "_cmpl" -> year -> album  ->  title = "f_browse"
"U"   -> albumartist ? albumartist ^ "U" | albumartist ^ "The U" & filename !~ "_cmpl" -> year -> album  ->  title = "f_browse"
"V"   -> albumartist ? albumartist ^ "V" | albumartist ^ "The V" & filename !~ "_cmpl" -> year -> album  ->  title = "f_browse"
"W"   -> albumartist ? albumartist ^ "W" | albumartist ^ "The W" & filename !~ "_cmpl" -> year -> album  ->  title = "f_browse"
"X"   -> albumartist ? albumartist ^ "X" | albumartist ^ "The X" & filename !~ "_cmpl" -> year -> album  ->  title = "f_browse"
"Y"   -> albumartist ? albumartist ^ "Y" | albumartist ^ "The Y" & filename !~ "_cmpl" -> year -> album  ->  title = "f_browse"
"Z"   -> albumartist ? albumartist ^ "Z" | albumartist ^ "The Z" & filename !~ "_cmpl" -> year -> album  ->  title = "f_browse"

the problem is albums are not filtered by year; whichever year i select, in the subseguent directory there are all albums of the artist

for example:
big thief ->
    2022 ->
        U.F.O.F.
        Dragon New Warm Mountain I Believe in You

but only the last one is from 2022.

What i'm doing wrong?
Thanks in advance

EDIT:
maybe it's important: i use a FiiO M3K with last dev builds installation of rockbox.

[Bilgus]

I can reproduce the bug but I haven't narrowed down why yet

[ilmioalias]

OK, thanks.
Shall i open a bug?

[Bilgus]

Well the issue appears to be with the logical OR
"L"   -> albumartist ? albumartist ^ "L" -> year -> album -> title = "fmt_title" Works

I'll try to figure out the issue but the tagcache is a big ole mess

[Bilgus]

I have a patch that fixes this https://gerrit.rockbox.org/r/c/rockbox/+/4248 Tagcache Don't create filters when parsing a logical OR

https://www.mediafire.com/file/12kxvxrq2qdlacn/FioM3kBaremetal_tagcache_bugfix4248_rockbox-full.zip/file

Should be in latest dev builds tomorrow..

[ilmioalias]

I have a patch that fixes this https://gerrit.rockbox.org/r/c/rockbox/+/4248 Tagcache Don't create filters when parsing a logical OR

https://www.mediafire.com/file/12kxvxrq2qdlacn/FioM3kBaremetal_tagcache_bugfix4248_rockbox-full.zip/file

Should be in latest dev builds tomorrow..

Thanks, it works!

[Bilgus]

Good, thanks for testing

[ilmioalias]

Good, thanks for testing

Unfortunately, the problem reappeared, but only for some combination of letters; for example some artists show the bug, but others don't

[Bilgus]

could it possibly be a formatting error for some artists? could you try the ones that work and rename them to names that don't?

[ilmioalias]

could it possibly be a formatting error for some artists? could you try the ones that work and rename them to names that don't?

You are right, there is a problem with artists names that start with "the"; in this case filtering doesn't work.

[Bilgus]

looking at your original sample above I'd say likely the really long line is blowing the stack
but really IDK, the tagcache code is complicated

[amachronic]

The problem here is how the tagtree constructs database queries. Due to the way the tagcache evaluates queries it can only deal with boolean expressions in "disjunctive normal form", which have this general form:

Code: [Select]

(a0 && ... && aN) || (b0 && .. && bN) || ...
or at least, "clauses" are dealt with in this fashion. This is further complicated by "filters" which (I think) become an additional AND at the outermost level but idk.

Each "level" of the menu in the tagtree corresponds to a new query (so albumartist -> year -> album -> title has four levels). When you are viewing the albums under a certain year you're at the 3rd level, and the tagtree will add clauses and filters corresponding to your selections at the 1st and 2nd levels:

Code: [Select]

FILTERS
  albumartist = "The A.."
CLAUSES
  year = 2005
(For some reason the tagcache cannot "filter" on numeric tags, so the year becomes a clause.)

Now to ensure it only finds stuff matching your actual query, the tagtree will append the entire conditional from the tagnavi file:

Code: [Select]

FILTERS
  albumartist = "The A..."
CLAUSES
  year = 2005 & albumartist ^ "A" | albumartist ^ "The A"
Now it is easy to see why the year gets ignored: the AND applies only to the left side of the disjunction and the right side is always true because we selected an albumartist that begins with "The A" already. If we selected an albumartist beginning with "A" then the query would work correctly because the right hand side would be false, forcing the year check to be taken into account. If you swap the order of the terms in the tagnavi file, like this:

Code: [Select]

"A"   -> albumartist ? albumartist ^ "The A" | albumartist ^ "A" -> year -> album  ->  title = "f_browse"
then the tagtree's final clause will end up being

Code: [Select]

  year = 2005 & albumartist ^ "The A" | albumartist ^ "A"
which make the menu work properly for albumartists beginning with "The A" but break all albumartists beginning with "A".

It's not too hard to fix the tagtree in theory but I won't be able to spend time on it anytime soon. Bilgus maybe you'd like to take a shot at it?

[Bilgus]

Looked at it poked at it came away with:
Above my pay grade

[Bilgus]

I think I've a solution, add a new clause operator
'@^' begins with one of

Code: [Select]

"L" -> albumartist ? albumartist @^ "L|The L" & filename !~ "_cmpl" -> year -> album  ->  title = "f_browse"
Edit:
Its in https://git.rockbox.org/cgit/rockbox.git/commit?id=ddbca125a6

[Bilgus]

@ilmioalias

when all is said and done could you post that script to the wiki, or even here to the forum so someone else might use
of all this time we've spent

Ps can 'T' be shortened to this?

Code: [Select]

"T"   -> albumartist ? albumartist ^ "T" & albumartist !^ "The " & filename !~ "_cmpl" -> year -> album  ->  title = "f_browse"