Tagnavi_custom.config: searching by comment field

[SoulSkorpion]

I've got the following in my tagnavi_custom.config:

%menu_start "custom" "Smart Playlists..."
"Games" -> album ? comment ~ "" -> title = "fmt_title"
"Not games" -> album ? comment !~ "" -> title = "fmt_title"

I've done this because I'd previously used tags in the comment fields like this on my old iPod, to generate smart playlists based on them. I've now got a Sansa e280, and I was hoping to get the same effect using Rockbox.

In any case, this isn't completely working. The first one works fine - there aren't any entries returned that aren't games. The second one is showing up entries that are games. When I find the files through Windows and check the tags in Winamp the "<game>" tag is still present in all the songs that incorrectly appear in the "Not games" list. Once or twice I checked the ID3 tags in Rockbox's online ID3 tag viewer, and the comments fields seem to contain garbage.

I've tried both initialising and updating the database (yes, leaving it on for 15 minutes so that the disk activity icon goes away) and it hasn't cleared up the problem. Is there anything else I can try? Is this a bug/limitation of the database code? (If so, any hints about where in the source tree to look would be appreciated and I'll have a crack at fixing it myself).

PS: sorry about posting twice in quick succession, but I figured it would be better to open separate topics for separate questions.

[roolku]

So if I understand you correctly, you own files that are tagged with "<games>" (as proven by using winamp) and Rockbox doesn't extract the tag info correctly (shows garbage)?  

If so, could you upload an example somewhere and post the URL.

Cheers
  Roolku

[SoulSkorpion]

Uploaded a sample to here.

I did some experimentation. My settings are to prioritise ID3v2 first, so I thought that maybe the files were only tagged in ID3v1 format. Opening it in Winamp, I found that it was tagged only in ID3v2, so that didn't help.

So, with ID3v2 only, looking at the ID3 details on my device through Rockbox shows the comment field as a series of digits (I think they were just zeroes, ones and twos).

Having told Winamp to enable ID3v1 and copy the values across (in case it fixed the problem, which iit didn't) I noticed that, in Rockbox, the tag was showing up as "ÿgame>" instead of "<game>" (thanks to Wikipedia's edit function for giving me a place to copy and paste that character from ).

[roolku]

Uploaded a sample to here.

Okay. The files contains two comment fields:

1) (iTunNORM) 00000119 0000014A 00000ED5 00000722 0000755E 00007547 0000419A 000037FC 00015FD5 00002755
2) <game> <strategy>

and rockbox uses the first, while tagging programs either ignore the COMM tags with an extra label (winamp) or use the second/last tag (tag&rename).

I prefer winamp's approach as the tags might be in different order and the same problem would show and I will prepare a fix when i have the time (probably tonight).

[Llorean]

And what if someone wants to use parenthesis in their comment? I'd rather it simply use the first comment, and people insure they know what's in their comments, rather than assuming parenthesis mean "skip this comment"

[roolku]

And what if someone wants to use parenthesis in their comment?

Then they should? Why would that be a problem?

I'd rather it simply use the first comment, and people insure they know what's in their comments, rather than assuming parenthesis mean "skip this comment"

So far I haven't seen a single tagging program that can show more than one comment not to mention one that allows to specify the order in which they are saved to the binary? Could you point one out to me please?

[Llorean]

Well, you said the first tag has (iTunNORM), and described that as an extra label. Did you mean something other than it having parenthesis in it as being the 'extra label' because I can't see how I could use (blah) in the beginning of a tag safely if you had it ignoring those tags.

And by "insure they know what's in their comments" they could perhaps only have one comment. I don't see how ignoring tags based on a non-standard methodology is a good idea. If there's no such thing as a tagging program that lets you order your comments (and I'd assume that there has to be at least one that lets you edit multiple comments if multiple comments is indeed valid, and I can only assume then that this one would write the comments in the order they're written) then wouldn't a proper solution be something like a plugin that lets you reorder your comments rather than arbitrarily ignoring a comment based on its contents?

[roolku]

Well, you said the first tag has (iTunNORM), and described that as an extra label. Did you mean something other than it having parenthesis in it as being the 'extra label' because I can't see how I could use (blah) in the beginning of a tag safely if you had it ignoring those tags.

No, the iTuneNORM text string is in the label field (also called shorttext) of the ID3 comment tag, which describes what the comment is about.

This shorttext is hardly ever used and again I don't know of any tagging programs that even expose it to the user for editing, however itunes uses it. It stores the normalisation information in a comment tag which is labeled iTuneNORM. Considering this is a problem for all users who use iTunes normalisation, comments and the database I think this is the solution that will cause the least confusion. I can't see any point in ever displaying the technical information from iTunesNORM in Rockbox.

And by "insure they know what's in their comments" they could perhaps only have one comment.

Well the two programs I use for tagging will only expose one comment tag for editing and leave any existing other ones untouched.

I don't see how ignoring tags based on a non-standard methodology is a good idea.

I don't think it is any more non-standard than ignoring the second tag like you suggested? I would even call it de-facto standard (as winamp's behaviour is often referred to like that).

If there's no such thing as a tagging program that lets you order your comments (and I'd assume that there has to be at least one that lets you edit multiple comments if multiple comments is indeed valid, and I can only assume then that this one would write the comments in the order they're written) then wouldn't a proper solution be something like a plugin that lets you reorder your comments rather than arbitrarily ignoring a comment based on its contents?

No, I prefer a KISS approach that is current practice in other programs to anything that makes it harder for the user. Also it is not ignoring it based on the contents, but on the fact that it is labelled. If you can show me any evidence that the labelling mechanism is commonly used for user-added comments (rather than for tagging technical information), I will reconsider.

[Llorean]

Well, part of the problem is the misunderstanding. You didn't clearly label "iTuneNORM" as being in another part of the tag. You referred to it as another label, but wrote it as part of the actual tag contents.

If it's not part of the tag contents, then I see no problem using it as a factor for ignoring tags.

But "Winamp does it" is not an excuse for why Rockbox should do something. Winamp is far from the only program out there. In this case, I assume the shorttext is meant for identifying the content-type of Comments, so ignoring ones that aren't simply "comments" as it were makes sense. But you still have to pick one if there are multiple that don't have shorttext, don't you?

The whole problem stems from it being unclear that (iTuneNORM) wasn't in the actual comment in your first post, which is why I made all those points about parenthesis.

[polygonal]

So far I haven't seen a single tagging program that can show more than one comment not to mention one that allows to specify the order in which they are saved to the binary? Could you point one out to me please?

Mp3Tag shows the multiple comments under names like COMMENT ITUNESNORM, etc.; foobar2000 also shows multiple comments - if I remember correctly, by combining them. I don't think either of them allow you to specify the order, though, and I don't think fb2k writes multiple comments at all. My humble opinion is that ignoring the labeled comments in the ID3 screen are the way to go, since I personally don't know of any program that write labeled comments rather than unlabeled, and it should provide compatibility for the unfortunate victims of itunes' unasked-for comments.

Also maybe not parsing the labeled comments as well (or an option to do it)? Because I ran into problems with truncated tag info in long tags, and my problem is solved in most cases (I still have some unusually long tag that's my own fault) by removing the extra comments left by itunes.

[SoulSkorpion]

So, does this mean Winamp has incorrectly tagged my files?

On my machine, both comment fields are showing up as " " in Winamp.

Any ideas why the one tag field is showing up in "ÿgame>" instead of "", in Rockbox?

[bascule]

Any ideas why the one tag field is showing up in "ÿgame>" instead of "", in Rockbox?

That particular one looks like a unicode-type error. Maybe that particular tag has been written as UTF-16 instead of UTF-8 or some other similar circumstance? My knowledge of the intricacies of unicode is not great.

[roolku]

Any ideas why the one tag field is showing up in "ÿgame>" instead of "", in Rockbox?

Your example file doesn't seem to have the problem?

[roolku]

Rockbox now (r14294) ignores "iTunNORM" ID3V2 comments. You will have to rebuild the database. Remember to backup run-time data.

[SoulSkorpion]

Any ideas why the one tag field is showing up in "ÿgame>" instead of "", in Rockbox?

Your example file doesn't seem to have the problem?

Well, it does on my Rockbox (that's why I uploaded it . Does it show up OK on yours? What version are you running?

[edit]I'll try the latest build and see if it fixes things...[/edit]

[edit again]It hasn't. The comment tag still begins with control characters that aren't meant to be there. I've just noticed that the "genre" tag also contains a control character.

I don't think the iTunes ID3v2 thing is the problem. I was only seeing this effect when there was only an ID3v2 tag present (with no IDv1) - adding IDv1 tags has changed the nature of the problem to the current "corrupt first character" problem. I've set Rockbox to prioritise ID3v2 over ID3v1, so I'm not sure why it's not picking up the fixed iTunes/winamp tags... [/edit again]