Rockbox Technical Forums

Rockbox General => Rockbox General Discussion => Topic started by: lalittle on May 21, 2007, 07:33:33 PM

Title: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: lalittle on May 21, 2007, 07:33:33 PM
The ignore the patch is not included because the dev's rejected it, meaning that it is not likely to ever be included in SVN.


When you say "rejected," do you mean they decided it wasn't something that they thought was necessary?  If so, that seems really odd to me.  I'd think that EVERYBODY who uses the database view would want this.

I tried to find out more information on this, but I couldn't -- it's hard to search for issues where one of the search terms is the word "the."

Thanks,

Larry
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: magnumforce2006 on May 21, 2007, 09:52:20 PM

The ignore the patch is not included because the dev's rejected it, meaning that it is not likely to ever be included in SVN.


When you say "rejected," do you mean they decided it wasn't something that they thought was necessary?  If so, that seems really odd to me.  I'd think that EVERYBODY who uses the database view would want this.


I'm puzzled about this as well  ???
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: Llorean on May 21, 2007, 10:13:11 PM
It's not the right way to solve the problem.

The proper solution is this:
There are in many tagging formats two tags each for Title, Artist, and Album. One for display, and one for use in sorting. If you want The to be ignored, for the sorting tag, you put "Blah, The" instead of "The Blah" (which you use for the display tag).

So a correct solution would be for someone to make a patch to use all of the tags effectively, rather than for it to intentionaly ignore a string of characters.

Until then, simply tag your files "Blah, The" if you want "The" not to be used in the search.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: magnumforce2006 on May 21, 2007, 10:14:22 PM

It's not the right way to solve the problem.

The proper solution is this:
There are in many tagging formats two tags each for Title, Artist, and Album. One for display, and one for use in sorting. If you want The to be ignored, for the sorting tag, you put "Blah, The" instead of "The Blah" (which you use for the display tag).

So a correct solution would be for someone to make a patch to use all of the tags effectively, rather than for it to intentionaly ignore a string of characters.

Until then, simply tag your files "Blah, The" if you want "The" not to be used in the search.


This is not the proper formatting for those artists, however, and will screw up many last.fm submissions.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: Llorean on May 21, 2007, 10:20:57 PM
Then write the patch to use both the Sort and Display tags.

A halfway solution should not be accepted just because nobody who wants to use it can be bothered to do it the right way.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: magnumforce2006 on May 22, 2007, 10:16:57 AM

Then write the patch to use both the Sort and Display tags.

A halfway solution should not be accepted just because nobody who wants to use it can be bothered to do it the right way.


And who deems this the "right way?" Why is shifting around a "the" field somehow sacrilege? I just don't really understand your reasoning.

Additionally, what do you even mean by sort and display tags? ID3V1 vs V2?
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: Yotto on May 22, 2007, 02:06:17 PM
Because it's arbitrary, and only works for one language (or maybe a couple, I'm not language... ologist).  Are you willing to code for Spanish, French, Italian... Maybe have a section where you can choose the language?  What about people who have multiple languages in their song titles?  What about Americans who would never be able to find "Bamba, La" because they don't think of that as the song title?

That's the first problem I can see, thinking about this for 5 seconds.  The "Right way" Llorean's talking about isn't "Keep all your songs that start with 'the' together," it's "Think up a more intelligent way to sort the songs that fixes the problem."
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: lalittle on May 22, 2007, 03:05:15 PM

It's not the right way to solve the problem.


So you're saying that while you're in favor of (or at least don't object to) the behavior that's being sought, it's the particular method that you disagree with, correct?

I'm still confused about what it is you dislike about the concept of "ignoring" certain text strings (plus one space) that appear at the beginning of the tag.  Isn't this literally the concept behind what needs to be done in order to sort the tags properly?  In JR Media Center, the option for this is called "Ignore articles (a, an, the)," and it works perfectly.  The articles are still present, but the sort order is correct.

Thanks for clarification,

Larry
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: magnumforce2006 on May 22, 2007, 03:30:53 PM
All I'm saying you guys, is that the default apple firmware and my old rio firmware, AND the creative players (I believe) all have databases with this sorting behavior.

It's what a lot of people are accustomed to, and I don't quite understand the resistance against it... Different languages? How would this adversely affect those users? And to add on to all of that... why not just make it an option? If you don't want that sorting behavior, don't use it.

I'm not a programmer so I'll stop there, because I know I really have no leverage in arguments due to that fact.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: Llorean on May 22, 2007, 04:31:26 PM
Your method: Clutters the code to solve the problem for ONE language in a firmware that supports many, and it does so in a manner that can conflict if for some albums you consider "The" a valid part of the sorting, while for others you don't (this is possible in many music collections).

My method: Works for any language or combination of languages, uses existing standards, and adds greater flexibility up to and including adding in one swoop the possibility to ignore The, ignore any other part of the title you want, have albums by various artists show up as a single album, have two disk sets have their disks labelled properly but still show up as a single album, all with one "feature".

The Rockbox developers decide what the "Right" way is, and reject patches that aren't the "Right" way because the patch tracker is the ROCKBOX patch tracker. It's for submitting patches to be included, and if we don't want them, they get rejected and closed. And by "Right" I mean "The way that is Right for Rockbox." Just because someone else chose to do it a different way is absolutely NO argument for Rockbox to use a halfway measure.

And no, I do not mean "ID3v1 vs ID3v2". Do some research in the tagging formats, or read the explanation I've given several times before when this same subject has come up. I'm getting tired of repeating myself on the same topic. See information on the TSOP, TSOT, and TSOA tags for ID3v2.4, and their equivalent tags in other tagging specs.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: lalittle on May 22, 2007, 06:02:46 PM
I'm pretty convinced I'm missing a fundamental piece of the "The" issue here because I'm honestly confused by some of the points being made.

Quote
Your method: Clutters the code to solve the problem for ONE language in a firmware that supports many,


I don't understand what constitutes "clutters the code."  Why is this particular feature considered "clutter" any more than any other aspect of the code that doesn't necessarily apply to everybody?  Rockbox contains lots of features that are of absolutely no interest to me personally, but I don't consider these things to be "clutter" just because I don't use them.

Also, why couldn't the patch be made to work in any language?  This is what other programs do.  Isn't it just a matter of adding a little extra code so that Rockbox takes into account what language you're using and acts appropriately?  It only has to do this for one to three words (the, a, an).  If the patch did this for all languages supported by Rockbox, would this make it a viable patch to the developers?

Quote
and it does so in a manner that can conflict if for some albums you consider "The" a valid part of the sorting, while for others you don't (this is possible in many music collections).


It seems really odd to give this weight when the VAST majority of artists/albums/songs will not conflict with the idea of sorting without the "The."  I assume that there are such examples, but I can't think of any, and I find it strange to give the .0001% of albums so much weight as to use it as one reason to reject this feature.  That said, nobody is saying that this feature shouldn't be disable-able for those that don't want to use it.

Quote
Just because someone else chose to do it a different way is absolutely NO argument for Rockbox to use a halfway measure.


Isn't it?  Doesn't the fact that so many other applications DO offer this feature essentially imply that a LOT of people want and use it?  The argument is not "others do it so we should also."  Rather, the argument is "others do it because it makes sense."  The reason iTunes and Media Center and iPod etc. offer this feature is because the commonly accepted standard is that you don't utilize articles (the, a, an) at the beginning of a "title" when sorting.

That aside, isn't the idea of Rockbox to offer MORE features in this regard -- i.e. to allow users to do things that you wouldn't otherwise be able to do?  I understand that the Rockbox developers get to make these decisions, but don't the general public's opinions factor into this?  I think this is where I'm missing something, so forgive me if the answer to this seems obvious to anybody.  I'm just trying to fully understand the conversation.

Llorean -- are you saying that we should tag material with "Name, The" rather than "The Name"?  It sounded like you were saying we should only do this as a temporary fix, but I'm not clear on this.  I strongly disagree with the idea of tagging everything with "Name, The."  Artists/albums/songs should be tagged as they appear on the album, which means not having to use the "comma" system to get them to sort properly.

Another thing I'm confused about is the idea of "sort" and "display" tags.  Are you saying that this is part of the native tag structure -- i.e. is this the way programs like iTunes and Media Center already work, or is this something that has to be "done" in order to utilize this feature?  If this is already part of the way it works, than I would agree that the patch should utilize this.  If this is something that has to be done, however, it seems like a moot point.  The sorting should work on tags in their "native" structure -- i.e. it should work on the tags that the popular programs create by default.

I apologize if this has been discussed before, or if I'm missing something that seems obvious to others.  I did try searching the forums, but I was unable to turn up the answers I was looking for.

Thanks again for any clarification,

Larry
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: Llorean on May 22, 2007, 06:17:52 PM
"Something else does it this way" is not an argument. This is final. Rockbox has stated this several timed: We do things the way WE think is the right way.

Just because the majority wants it doesn't mean we should do it. WE are writing the software, not you. You don't pay for it. You don't buy it. You have NO pull in the direction it goes. What gets done is the way the people writing the software feel it should work. They chose to reject the patch because they think it's the wrong solution. The majority has wanted Album Art for a long time. The patch has existed for a long time. It has worked, reliably, for a long time. You'll notice it hasn't been committed because it's not the RIGHT way yet. Rockbox is given away free, and things are done to please the users, but you are not customers. The designers don't depend on you, and if they feel something doesn't work the way they want it to, they won't do it no matter how much you pester or complain (see the Volume % vs Volume dB debate).

Files have two tags for Artist. A Display tag, and a Sort tag.

Display could say "The Name", and sort could say "Name, The". You would never, ever SEE "Name, The", but when the list is organized, "The Name" shows up where "Name, The" fits in the list, because it sorts by the sort tag, and displays the display tag.

iTunes does not make use of this tag, but it's a part of the several standards, and solves FAR more problems, without having to make arbitrary decisions as to what to ignore.

And "Clutter" is code that does something that a user can avoid easily just by having their files properly tagged. "Clutter" is a solution that solves only an aspect of a problem in a manner that can create other problems.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: lalittle on May 23, 2007, 02:41:32 AM
Files have two tags for Artist. A Display tag, and a Sort tag.

Display could say "The Name", and sort could say "Name, The". You would never, ever SEE "Name, The", but when the list is organized, "The Name" shows up where "Name, The" fits in the list, because it sorts by the sort tag, and displays the display tag.


I've never heard of this being part part of the ID3v1 and/or ID3v2 mp3 tagging standards before, so I didn't know about this.  That said, I'm confused how/where the "sort" tag ends up set to "Name, The" instead of "The Name."  How is the order of words in the "sort" tag defined?  You're not advocating that users set this manually, are you?

Thanks,

Larry
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: Llorean on May 23, 2007, 09:58:01 AM
Yes, I am. It's a user's responsibility to properly tag their files. Or a tagging program could handle this automatically if a good enough tagging program existed, and its authors didn't mind that this solution only applied in English, and could potentially be disastrous for non-English titles if processed at the same time.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: DefineByte on May 23, 2007, 10:24:52 AM
So TSOP is meant for the artist? The spec doesn't say that; or is performer analogous to artist? Why the need for both then?
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: Llorean on May 23, 2007, 12:52:01 PM
Yes, TSOP should be used for "Artist" when sorting. In terms of ID3 tags, the Artist tag is actually the "Performer" tag.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: DefineByte on May 23, 2007, 01:21:22 PM
Thanks.

I'm tempted to ask you all sorts of ID3 questions but I'll refrain. :D
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: Llorean on May 23, 2007, 01:26:13 PM
I'm hardly an expert on tagging. I just do research when a subject comes up, before I start on discussing the subject, and this is research I did months ago when the topics of "Albums with various artists" and "Sorting in the database" were discussed elsewhere.

Using the three stort tags would solve several problems at once, in a 100% multilingual way, without obvious negative side effects.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: lalittle on May 23, 2007, 03:16:50 PM
Using the three stort tags would solve several problems at once, in a 100% multilingual way, without obvious negative side effects.


I guess the problem is that I honestly don't understand how a solution that requires the user to manually edit the tags on every artist, album, and song that starts with the word "The" is considered a "solution" when it requires a notable amount of user intervention.  This is especially true given that it would require most users to use an entirely new program since many if not most of the popular media management programs (iTunes, Media Center, etc.) don't allow access to the "sort" tags individually.  I think what's needed is a way of handling this with as little user intervention as possible.

That said, what if the sort feature:

1)  Sorted by ignoring articles at the beginning of the tag, UNLESS

2)  it saw a specific character pattern at the beginning of the "sort" tag (say a double tilde "~~" for example.)

This way, ONLY the "special case" artists/albums/songs would have to be manually edited -- i.e. you'd only have to add the double tilde to the beginnings of "sort" tags where you WANTED them to sort WITH the articles.  Since this is a such a small percentage of the artists/albums/songs, this would eliminate a HUGE portion of the tag editing necessary, and for "most" artists/albums/songs it would work "by default" with no intervention needed.  It would also only require the use of a new tag editing program for people that had "special case" titles in their libraries.

3)  The behavior could be turned off.

4)  The patch would be made to work on a "per language" basis, where it only ignores articles in the language currently being used.  This could either be done by have the patch check for what language is currently being used, or by allowing the user to select the language that this feature effected.

This would take care of the language situation, it would not require the editing of the majority of tags, it would sort "special case" titles correctly, it would not require iTunes or Media Center (or other programs) users to use another tagging program, and for "most" artists/albums/songs it would work "by default" with zero invtervention needed.

Would this be an acceptable solution to the developers?  It addresses all the issues brought up without necessitating a lot of tag editing, which would make it accessible to everybody, including people that didn't know about a seperate "sort" tag (which until I read this thread included me.)

Thanks for your feedback on this,

Larry
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: DefineByte on May 23, 2007, 03:54:58 PM

I'm hardly an expert on tagging.

I don't think anyone is. Disc subtitles, for instance, seem to flummox a lot of people.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: Llorean on May 23, 2007, 04:50:51 PM
Why do you suggest doing it in such a huge hack of a way? Sorting without articles by default?

I've told you: THAT IS NOT THE DESIRED BEHAVIOUR. You're not going to convince me otherwise, so please stop trying.

Please, if you're going to continue beating your head against this wall, do it elsewhere. Convince Apple to add a feature into iTunes that duplicates the "Performer" tag into the "Performer Sort Order" tag, and removes the article.

A list of articles for each language is just a worst hack.

Again, I don't see how "My tags are bad" is an argument for "The Rockbox guys should do something they don't want to do."

YOU can fix your tags. We have our software how we want it. Why exactly do you think we should make something work a way other than how WE want it, when it's software designed for us, by us, and you don't pay for it?

Remember: You aren't the consumer here. Rockbox, ultimately, doesn't depend on users. Every now and then a decision will be made, and it's not a democracy. There are a few people (I'm not one of them) who have the absolute final say on how things go. Decisions get made, and in some cases people don't agree with them. You aren't paying the bills, and you're free to use any other firmware of your choice. Rockbox isn't invested in keeping you as a customer.

It would be nice if you stuck around, but keeping you around is not a reason for the core Rockbox devs to implement a feature that they feel isn't the right solution to the problem.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: lalittle on May 23, 2007, 09:16:05 PM

Why do you suggest doing it in such a huge hack of a way? Sorting without articles by default?


I respect your request that I not beat this into the ground, but since you specifically asked, I'll answer this question.  Please understand that I'm not doing this in order to try and "convince" you of anything -- I'm simply trying to communicate my ideas on this subject more clearly.  I thougth this this may still be of interest to you regardless of what the end result is.

What we're disagreeing on is the set of parameters that are important in this feature, and what constitutes a "hack."  To me, one of the important aspects of this feature is that it should do the sorting without user intervention -- i.e. that it should not require a separate process of editing tags in order to achieve this behavior.  I understand that you don't agree that this is important, but it is important to me.  There is no "right" or "wrong" in this regard -- it's based on subjective opinion.  From my point of view it's therefore not a "hack" since your method does not achieve what I'm looking for -- it just doesn't address a specific condition that I see as important.  This NOT to say that you're method is "wrong" -- it simply is not a solution given my parameters, which are different than yours.

In other words, while we're both talking about the same END result, we're really not talking about the same thing.  To me, the process needs to include the concept of working with the NATIVE tags from popular programs.  Without this extra idea, the process doesn't achieve the goal that I'm personally looking for.  I fully understand that your method satisfies your needs, but our needs/desires are different.

I don't know enough about this to do it myself, but hopefully, others will continue to work on a patch that achieves the goal without requiring tag editing.  There is really no reason we can't both have the behavior we're looking for -- I may just have to use a patch that is not officially endorsed by the developers.

Thanks,

Larry
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: DefineByte on May 24, 2007, 04:29:37 AM
It should be easy enough to write a script in foobar2000 to create the TSOP tags for you (foreign languages not withstanding). Is the extra effort your only beef?
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: GodEater on May 24, 2007, 04:53:29 AM
From what I can gather, it seems *we* should implement a crappy solution in our software because either :

a) He can't find a tagging program that does the sort / display stuff
or
b) He can't be bothered to learn about one that does
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: lalittle on May 24, 2007, 07:45:22 AM

From what I can gather, it seems *we* should implement a crappy solution in our software


That's not fair.  I'm not saying you should "do" anything -- I'm stating an opinion that you happen to disagree with.  By having a discussion about this subject we can potentially understand something about the other's point of view.  Regardless of whether or not any body's opinion is changed, it's still productive to try to understand other points of view.  Calling it a "crappy" solution serves no purpose -- it's only creates an adversarial tone that makes it more difficult to have a meaningful conversation.  We can respect each other's opinions even if we don't agree.

Quote
because either :

a) He can't find a tagging program that does the sort / display stuff
or
b) He can't be bothered to learn about one that does


What's wrong with not wanting to be "bothered" with another step, even if it's a relatively simple one?  It still takes extra time, so it's still more "convenient " to not have to do it.  For example, I like being able to simply hit "record" on a TV program in order to record it rather than having to manually input record start/stop times.  It takes more code and an entire system to support it, but it's a lot more convenient to the end user.

I seen nothing wrong with wanting the software to accomplish "article free" sorting without requiring my intervention.  It's not because I'm "lazy," it's because it's more convenient to not have to go through the extra tagging step.  Please note that I'm NOT trying to convince you to do this -- you've made your position perfectly clear. I'm just responding to your post, which I personally did not think was a fair representation of my opinion.

From my point of view (which I fully understand is different than yours), eliminating the need for user intervention would be a fantastic aspect of this feature.  I merely expressed my opinion on this subject on an open forum where user feedback and opinions are welcomed.  It's one thing to disagree with me -- even strongly -- but please don't ridicule my ideas just because my opinions differ from yours.

As to your two conclusions,

You're correct that I don't want to have to use a separate program to tag my files.  I want to continue to use Media Center for my all my media needs (tagging, management, playback, etc.)  Even if Media Center DID offer the ability to edit the "sort" tags separately (which it does not), I would still rather not have to do this.  I don't call this laziness -- I call it wanting a more convenient solution that requires less work on my part.  It's simply about wanting a more robust tool.

And just to be totally clear -- I FULLY understand that the Rockbox developers do NOT have to take ANY of my suggestions to heart.  These forums are for the free exchange of ideas, and I'm simply offering my opinion for anyone interested.  When it comes to a project like this, I firmly believe that offering opinions and ideas are the cornerstone for positive developments.

Thanks,

Larry

Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: TedGamble on May 24, 2007, 07:52:09 AM
For all you folks that are still depending on iTunes (or any other tagging software that falls way short of your expectations in terms of features), I have two words:  THE GODFATHER.  This FREE (my favorite F-word) software is loaded with just about every feature that you can imagine:  Batch tagging, batch rename, regular expression matching, and much much more.  Get this program, tweak your tags (in batch) and lay off beating up the Rockbox team because they don't want to do it your way.

http://users.forthnet.gr/the/jtclipper/

And, by the way, just because their website says "Beta", do not fear.  I've been using this fan-damn-tastik program for 6 months with NO, ZERO, ZIP, NONE problems.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: lalittle on May 24, 2007, 08:00:40 AM

It should be easy enough to write a script in foobar2000 to create the TSOP tags for you (foreign languages not withstanding). Is the extra effort your only beef?


Thanks for the suggestion -- I appreciate it.

Yes -- the extra effort is essentially my only beef.  Unfortunately, extra effort and time would still be needed even with a foobar script.  Having to use foobar -- even with a script -- would require extra time that I just don't have.  I know this may seem strange to some people who do this sort of thing all the time, but I have a limited amount of time to spend on this sort of stuff, and I'm already WAY over my limit.  My preference is still for a system that takes care of this automatically.  Others here call this a "hack," but to me, it would constitute a "solution" given the parameters that are important to me.

That said, if a script was written for foobar that created the "sort" tags automatically from the existing tags, wouldn't it have to use the EXACT same logic that I was talking about Rockbox using?  Rather than having a program ignore articles directly, we'd have a situation where one program rearranges the words in the tag in such a way that another program sorts them correctly.

Thanks for the suggestion,

Larry
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: lalittle on May 24, 2007, 08:08:46 AM
lay off beating up the Rockbox team because they don't want to do it your way.


I NEVER intended to "beat up" on the Rockbox team in ANY way -- I honestly hope my comments were not taken this way (although I fear they may have been based on some of the reactions.)  I appreciate everything the developers have done with this project, and I have a great deal of respect for their expertise and dedication to Rockbox.  They've put a LOT of time into this, and they've done an incredible job.  I'm simply voicing my opinions and ideas on this subject since I truly believe that this can be helpful to a project like this.  I FULLY understand they may not agree with me, and I have no "expectations" with regard to them utilizing any of my input -- I merely offer my opinions for the purpose of giving feedback.

Larry
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: Llorean on May 24, 2007, 08:54:47 AM
Just as a note, I do fully respect your right to offer your opinions on things, and if I've come across as somewhat defensive I apologize. It was beginning to feel like an argument to me, since it felt you were repeating the same reasons for doing it as had been said before.

Yes, the exact same logic would apply in the foobar script. But the logic itself isn't purely bad: If your collection contains only English songs, with a set list of known articles, and all songs need the article removed, the logic is fine.

Having the logic handle more languages, or a longer list of articles (the list of articles skipped would, for a fully functional feature, need to be independent of the current display language) would be part of a "properly" working feature. As well, if the TSOP patch feature were included, having an article skip feature would be redundant on the assumption that a user has properly tagged files (Rockbox already makes this assumption in several other places, so please don't use the argument that a user may not have properly tagged files: if they don't, it's their choice, and their choice alone). There are many reasons not to waste binary space on redundant features.

In the end, the TSOP patch offers the greatest flexibility regarding this feature.

I know it's not ideal for what you want, but a batch script could be used with Foobar as has been said, and I'm not sure about the "I don't have the time" argument on that: If you convinced someone to make the batch script (and I'm sure someone will at some point if the TSOP tag support gets written) then it's a matter of a batch process. You leave your PC running, and it scans the performer tags, and generates TSOP tags without the article. Heck, someone could even write a plugin for Rockbox that would (much more slowly) generate sort tags off the existing tags.

In my opinion though, sort tags would be an "optional" feature. You add sort versions of the tags if, and only if, the "Sorted by" information should be different from the display information. So for all artists not containing an article, and all artists you wish to continue to include the article in the sort, you don't have to add additional tags. Just run the batch script overnight on the other artists while you sleep, and you're done.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: saratoga on May 24, 2007, 09:45:11 AM

It's not the right way to solve the problem.

The proper solution is this:
There are in many tagging formats two tags each for Title, Artist, and Album. One for display, and one for use in sorting. If you want The to be ignored, for the sorting tag, you put "Blah, The" instead of "The Blah" (which you use for the display tag).

So a correct solution would be for someone to make a patch to use all of the tags effectively, rather than for it to intentionaly ignore a string of characters.


I don't think adding more ID3v2 fields is really the best solution.  Besides the normal issues with incompatibility between custom fields across different software, theres also the practical problem of actually updating a large amount of music with custom fields potentially across many different formats.

IMO there should be a preprocessor that examines tags when they're loaded.  A language specific text file could define what strings should be parsed out, thus allowing intelligent behavior across many different languages while still allowing consistent behavior with other software.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: Llorean on May 24, 2007, 09:49:02 AM
They're not custom. They exist in several tagging specs including ID3.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: DefineByte on May 24, 2007, 10:36:40 AM
Code: [Select]

$if($strcmp($substr(%artist%,0,4),'The '),$substr(%artist%,5,$len(%artist%))', The')

Simple enough. Adapt as required (not that it's supported in rockbox yet).

I love foobar. :D
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: Dixie Flatline on May 24, 2007, 01:03:36 PM

They're not custom. They exist in several tagging specs including ID3.

This has been very educational, since I was unaware of the TSOP/TSOA/TSOT tags in ID3v2.4.  I'd agree that where this mechanism is available, it's the Right Way to solve the problem universally.  (Of course, I'm obsessive about tag maintenance on my own music collection, so I may be biased.)

However, 99% of my own collection is in either FLAC, Ogg Vorbis, or Musepack (transcoded from FLAC for use with Rockbox).  The former two use Vorbis Comments, the latter uses APEv2 tags.  Obviously, tag names in these formats are freeform, but I've been unable to find any recommendation for separate sort-order tags to use with those two tag formats.  Are the Rockbox developers aware of any?  I'd love to use them if they exist, but from what I can tell so far, they don't.

If they don't exist, then some sort of functionality similar to the "ignore the" patch would be a welcome thing for those whose tag formats don't offer the option of doing things the Right Way.  Certainly not as default behavior, and preferably not English-only, but if it were implemented as a display option, with some mechanism to configure the list of articles used (i.e., to allow localization), would that be more acceptable to the Rockbox developers?

IANAP, so all this is just a thought, but I am very curious to know whether the sort-order tags exist for Vorbis Comment and APEv2 tag formats.

EDIT: See my post below.  Looks like the sort-order tags do exist for Vorbis Comment, APEv2 and lots of other tag formats, so this whole argument becomes more or less inoperative.  Never mind...
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: elborak on May 24, 2007, 01:16:57 PM
Here is the best tag cross-reference I've been able to find: http://www.matroska.org/technical/specs/tagging/othertagsystems/comparetable.html

Another thing to keep in mind: no one is saying that you can't have this functionality in Rockbox, just that the current attitude among the core developers is that it is not the right approach for their codebase. It's pretty easy to produce a patch and do a local build with whatever customizations you want. Throw it up on a web page for others if there's enough demand. One of the true joys of the GPL.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: Dixie Flatline on May 24, 2007, 01:41:54 PM
Thanks.  The "IANAP" in my most is "I Am Not A Programmer" -- it's unlikely I'll be implementing it myself, but wanted to point out that some sort of similar functionality would probably be welcomed by a lot of people, if it could be brought into the trunk gracefully and non-intrusively.

However, it also looks like I should have done just a little more Googling before I posted.  Thanks for the chart from Matroska.  I also just found this: http://musicbrainz.org/doc/PicardQt/TagMapping, which suggests that ALBUMSORT, ALBUMARTISTSORT, ARTISTSORT, and TITLESORT are the appropriate tags for both Vorbis Comment and APEv2 formats.

Since this functionality seems to be better-represented across tag formats than I first thought, I withdraw the argument made in my previous post.  Since the tags are available to make custom sorting work, and we have the godlike powers of The Godfather and fb2k at our command, it does seem that getting Rockbox to recognize and honor the various sorting tags is a better way to solve the problem.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: saratoga on May 24, 2007, 02:47:06 PM

They're not custom. They exist in several tagging specs including ID3.


How much software support is there for them?
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: elborak on May 24, 2007, 02:58:52 PM
How much software support is there for them?
Assuming you mean tag editors (as playback software would be irrelevant to the current discussion), any that lets you edit arbitrary ID3 tags. Mp3tag would be one.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: lalittle on May 24, 2007, 04:57:29 PM
Heck, someone could even write a plugin for Rockbox that would (much more slowly) generate sort tags off the existing tags.


This, to me, would be a better solution than having to use foobar for this task.  I much prefer keeping all the Rockbox-related work within Rockbox itself since it does not necessitate using another, separate application in order to achieve this behavior.  Maybe it's largely a psychological difference to have the Rockbox "duties" confined to Rockbox, but for whatever reason it definitely makes a difference to me, and it would make the difference between me having "article free" sorting or not.

Just to give another example of the way my brain works on this stuff, I have all my cover art contained with the mp3's -- this is my preference based on what's important to me.  I realize that I can run a batch process on my files that would extract the cover art and create versions of this art that the Rockbox iPod could read and display.  I just don't want to have to take the time to do this, however, every time I rip a new album.  Therefore, I end up with no art in Rockbox.  I understand that this is a "choice" I'm making, but to ME -- to MY personal way of working -- it's not worth the trouble to go through this process in order to achieve the outcome.  I WOULD like to have the cover art, and if someone created a Rockbox patch to utilize the native artwork within the files, or that would do this process from within Rockbox, I would use it.  Until then, however, I'll just forgoe art in Rockbox.

All that aside, I agree with saratoga on this one, i.e.
Quote
IMO there should be a preprocessor that examines tags when they're loaded.  A language specific text file could define what strings should be parsed out, thus allowing intelligent behavior across many different languages while still allowing consistent behavior with other software.


This just makes "more sense" to my way of thinking -- even with the arguments against it, it still seems like the "right way" to achieve this behavior.  That's just my opinion -- I realize that it's not my choice to make.

Larry
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: Llorean on May 24, 2007, 06:15:06 PM
Okay, exactly why does the way that is entirely redundant seem the right way?

The TSOP features need to go in for a number of reasons:
1) They allow easy removal of any article the user chooses.
2) They allow easy sorting of accented characters (by typing non-accented characters in the sort-field)
3) They allow easy various artist support (by using "Various" in the sort field).
It also is just flexible and there are probably other common problems it solves. This basically should go in regardless.

Your method only works with a set list of articles, requires a list in every possible language that could desire to ignore articles, cannot be turned on and off by a song by song basis but rather ignores all articles or none, and does nothing to solve any of the other problems. As well, it would be entirely redundant if the other, imho essential, code went in.

What makes it "right" other than the fact that it requires no user intervention to set up? I think I don't understand what you see as the advantage other than it means users don't have to do a (relatively minor) extra step in making sure their files are properly prepared (a step that can be automated by any enterprising person).

Also, I don't even understand why eliminating the article makes a list easier to read, but you still want the article visible. This is somewhat unrelated, but to me

Bad Religion
Blue October
Blue Oyster Cult
Metallica
The Beatles
The Rolling Stones
Queen

looks a lot better than

Bad Religion
The Beatles
Blue October
Blue Oyster Cult
etc...

I mean, it's not like they aren't then in alphabetical order under the "The" section, and it presents a more unified look.

But I'm not arguing against the option to remove articles, it just seems silly to waste binary size on a very, very limited solution, and I don't understand why you think it's the "right" solution in the context of Rockbox. The only advantage I see to Rockbox is that it's slightly easier for users, but I think that's far outweighed by the related costs (binary size, since it would in effect be redundant and the fact that it depends on an arbitrary list).

I mean, I understand why you like it more: It's simple, and easy for you to use, and you relate yourself as the average user. But it needs to be fully effective for all users, with a minimum of impact on those who don't use it, and a maximum of flexibility for what is added. So while I don't have a problem with your solution, I don't see how you can state you think it's right for Rockbox, which is multilingual, aimed at (on average) somewhat advanced users, and wants to try to keep the binary size down. (As a note, I'm not certain, but what I know suggests adding 3 tags to the database will result in much less binary increase than a new sorting algorithm with associated menu options and list of articles to avoid).
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: lalittle on May 24, 2007, 09:31:03 PM
Also, I don't even understand why eliminating the article makes a list easier to read, but you still want the article visible. This is somewhat unrelated, but to me

Bad Religion
Blue October
Blue Oyster Cult
Metallica
The Beatles
The Rolling Stones
Queen

looks a lot better than

Bad Religion
The Beatles
Blue October
Blue Oyster Cult
etc...

I mean, it's not like they aren't then in alphabetical order under the "The" section, and it presents a more unified look.


I don't have time to address all your questions at the moment, but I wanted to address this one (I'll post more later.)

Basically, it comes down to preference, and what one "expects" to see based on what they're used to in other systems.  I "expect" to see "The Beatles" under "B," which is most likely due to the fact that this is the way other systems work.  It is a generally accepted standard for "collections" or "lists" to be sorted by ignoring leading articles.  Libraries, indexes, lists of movies, books, songs, etc. all tend to use this approach.  I guess the idea is to use the first "important" word when sorting rather than the very first word period.  This just makes intuitive sense to a lot of people, which I assume explains why it is in fact a standard.  For lack of a better word, it "feels" logical to do it this way.

I'm not sure if this explanation really helps or not (I hope it does at least a little.)  This is a highly subjective area, and it's somewhat difficult to explain the specifics of what makes one approach "feel" better than another.

Larry
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: lights0ut on May 24, 2007, 09:48:18 PM


Bad Religion
Blue October
Blue Oyster Cult
Metallica
The Beatles
The Rolling Stones
Queen


Bad Religion? Awesome! Let me guess, your favorite song is "I wanna Conquer the World"? ;) By the way, thanks for splitting this thread, Llorean,  it's much appreciated. To add my 2 cents, I'm leaning towards Llorean's opinion: The sorting should be done with tags, after all, that is how music is sorted, by looking at the tags. By keeping the sorting in the tags you get other features (like Llorean and others pointed out) like being able to have some artists sort under various. I really don't use database view (as most of my music isn't tagged), but I don't find it irritating that The Killers, and The White Stripes are both under T for The, it's always been sorted that way (on my PC and on my DAP) so I'm used to it. While I understand that 'ignore-the' is quick and painless for users, it's not so quick and painless for the rockbox codebase, and this is what matters most to the devs. This is why we have to be patient for things like album art (which I am not complaining about, just making a point).

In the end, I trust what the devs do with the code, the focus seems to be on a rock solid DAP, with awesome audio playback,  that has well implemented, and equally solid features, plugins, and extras.  (way more than 2 cents worth, I think)
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: elborak on May 24, 2007, 09:51:28 PM
Basically, it comes down to preference
Exactly, which is Llorean's point. Supporting the sorting tags allows everyone to sort exactly as they want according to their preference. The other approach satisfies your preference but leaves others without a solution. Hence the choice seems obvious.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: lalittle on May 25, 2007, 01:49:35 AM

Basically, it comes down to preference
Exactly, which is Llorean's point.


My point is that issues directly pertaining to the sort behavior are only PART of this discussion -- there are other factors involved, such as the conditions under which this sorting takes place.  Whether or not the process requires user intervention is an important aspect of this to me.  I understand that this isn't the case for everybody, but it is for ME.  You can accuse me of being "lazy" if you wish, but that doesn't change the fact that having to go through a separate process in order for this to work is a central issue for me, and THIS is at the core of my argument.

Quote
Supporting the sorting tags allows everyone to sort exactly as they want according to their preference.


But it does NOT do it automatically, which once again is an important aspect of this in MY personal opinion.  You can point out OTHER advantages to "your" method, but these have no bearing on the issue of making the feature fully automatic.  We are talking about two different aspects of one overall subject.

Quote
The other approach satisfies your preference but leaves others without a solution.


Remember, however, that the reverse is ALSO true -- i.e. your approach does not satisfy MY needs given that non-intervention is one of my "needs" in this situation.  We simply disagree on how much extra effort is "acceptable" in order to get this behavior.  I want to rip, sync, and go -- and have everything sorted without the articles.  Your method does not offer this.

Quote
Hence the choice seems obvious.


Your method is the obvious choice from YOUR point of view, but "my" method is a better choice from my point of view because your method does not satisfy certain parameters.

Keep in mind also that when you bring the idea of "others" into the equation as you did, you need to consider that we really don't know how many people would prefer one method over the other.  Your statement implies that I'm essentially alone (or at least in the minority) in my opinion, but the fact of the matter is that we don't know this.  It could be that more people would want to keep the process entirely automatic compared to those that would prefer the other capabilities that your method offers.

As pointed out by the developers, however, it doesn't matter -- it's up to them what they do, and I accept this (althougth I'll still voice my opinions on the subject.)  I'm simply saying that without a lot more evidence, it's not fair to paint a picture where my opinion is the minority one.  The opposite "could" be true.

Larry
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: Llorean on May 25, 2007, 01:56:50 AM
You can use MY method by performing one extra step: Tagging.

Others CANNOT do certain things with your method independent of the number of extra steps because it is inflexible.

This is a case of "Everyone can do anything, though some need a little extra work" vs "Some can do what they want with a minimum of work, and others can't do what they want at all" and I think the option that actually allows everyone to use the feature is preferable, clearly, because it doesn't force exclusion.

You cannot claim to be forcibly excluded: You CHOOSE not to fix the tags. Meanwhile, those who speak a language that Rockbox doesn't support the articles for, or those who have songs that are mixed case as to whether the article needs to be removed or not (multilingual: Something may be an article in one language and not one in another) are forcibly excluded by your method, and no choice of theirs can make their existing collection compliant with a limited system.

It's not a case of your opinion being the minority, I suspect the majority would prefer a simple method that doesn't require intervention. It's a case where yours forcibly excludes a group, your method is redundant (it makes the binary size bigger, but you seem to not care about technological hurdles in the slightest), and if we were to avoid the redundancy your method doesn't offer any solution to the other problems that the sort order tags solve (without coding independent solution, which brings us back to the binary size issue).
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: GodEater on May 25, 2007, 02:03:02 AM
@lalittle -> I'd also say, to be blunt, looking through this thread, that you've not gathered hordes of support for your "solution" - you do seem to be the only person arguing for it.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: lalittle on May 25, 2007, 03:00:33 AM

@lalittle -> I'd also say, to be blunt, looking through this thread, that you've not gathered hordes of support for your "solution" - you do seem to be the only person arguing for it.


I can't argue with that (although I'm technically not the "only" person on my side in this thread.)  Then again, only a handful of people have posted in this thread at ALL, so I don't think any conclusions can be drawn one way or another.  We just don't have a large enough cross section of opinions posted.

I may in fact be in the minority on this one -- I'm merely pointing out that this "may" not be the case, and I did this in response to the idea that one method would be more conducive to a larger group of people.

Larry
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: nls on May 25, 2007, 05:49:50 AM
I never use the database but personally I would not use the ignore article in it's less flexible incarnation, for example I have swedish artists starting with "the" that I would not want sorted withouht article but would like most of the american artist starting with "the" to be sorted without the article, also I would not like some of my artists with swedish names to be sorted without articles. IOW if this kind of flexibility was impossible I wouldn't use it at all.

That means that I agree with Lloreans suggestion using the sort order tags.
Another good point with this approach is that it would not require any additional options to turn it on or off, just sort by sort order tag if present, else sort by display tag.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: Rincewind on May 25, 2007, 09:16:09 AM
I want to add another point why only stripping certain articles from the sorting isn't enough.

With Jazz artists, it is a common sorting scheme to sort them by the last name.

e.g.
Code: [Select]
John Coltrane
Miles Davis
Jan Garbarek
Keith Jarett

This is the order they appear in a (my favorite) music store.

I would really like to have support for more tags in the database. It would be very nice to have a tagnavi.config where you can define (=use) all the tags you want to display, even custom ones. The tag scanner then uses this information to scan the additional tags that are defined (for whatever purposes). A common set of tags (used in the wps) should always be scanned, if they are in the config or not.

Advantages:
- every tag tag that someone possibly want to use in the tagnavi.config is available
- no database space is wasted, people who don't want fancy tags leave them out of their tagnavi.config and then they aren't scanned.
- no special cases need to be coded for album/track artist, sort fields, cd numbers etc. The tagnavi.config defines the use of special tags (some more syntax might be needed there).
Disatvantages:
- if the  tagnavi.config is changed, the database needs to be rebuilt.
- the structure of the database would be a little bit dynamic
- a host-based database builder needs access to the tagnavi.config

what do you think, is this worth a proper feature request?
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: alienbiker99 on June 10, 2007, 11:24:51 PM
if anybody is interested, norbusan edited this patch to support three other leading "the" in other languages. If somebody could edit the patch so that it can support ignoring two and one letter leading words, we could have it also ignore "a" and any other leading words such as "el" or "la"
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: lalittle on June 11, 2007, 02:14:10 AM

if anybody is interested, norbusan edited this patch to support three other leading "the" in other languages. If somebody could edit the patch so that it can support ignoring two and one letter leading words, we could have it also ignore "a" and any other leading words such as "el" or "la"


Thanks for the information.

As far as I know, the articles that are traditionally ignored in sort lists are "a," "an," and "the."  If these three words could be ignored in each of the languages that Rockbox supports, it would essentially take care of this specific issue.  It would of course not address things like "last name first" sorting, but I'm actually used to this at this point, so an "ignore" function would take care of everything I'd personally need, and it would not require any re-tagging, which is an important issue in my opinion.

Thanks,

Larry
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: bascule on June 11, 2007, 02:21:38 AM
How does this patch deal with different languages?

I remember on my Rio Karma that artists/albums beginning with 'Die' (to be killed) were mis-sorted because it was treated as the German for 'The' :-\

Can it be made to be dependent on the selected language?
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: safetydan on June 11, 2007, 02:52:45 AM
I decided to take a crack at supporting the sorting tags. You can see a very early version of the patch here http://www.rockbox.org/tracker/task/7287
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: seani on June 11, 2007, 04:41:02 AM
I'd  just like to +1 for sorting based on a separate, standards based, used-editable tag(s):

1) As stated a number of times, maintenance of tags is the users responsibility. There are swathes of free apps. that will let you edit tags, and a number that will let you copy a tag from one field to another. This would make it a relatively painless process to "correct" currently tagged files and keep on top of newly tagged files. From memory, TGF may have some support for stripping/transforming tags as they are copied in any case.

2) "Special" processing to press a single value into service in different contexts *always* causes problems, both from the messiness of the implementation, and the resultant functionality. This is a problem I've encountered time and time again in my own systems when a single description has to suit the needs of telesales, online sales, first and second line support, accounts etc. Each faction has it's own requirements in terms of formatting/sorting.

The sensible option is to have a separate related attribute (ID3 tag in this case), and have each interested party maintain the attributes in the way they see fit. This generally leads to resistance (as it's a maintenance "overhead" despite being "essential") and dissatisfaction with any method that tries to do this automatically by parsing the single value actually held (there are always ambiguities and odd cases). An unhappy compromise.

I don't always have the clout to insist this approach is followed, paying customers and all, but Rockbox has no such restrictions...
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: alienbiker99 on June 11, 2007, 03:28:42 PM
i dont want to in anyway steal norbusan's work and the others of the patch so ill link to where he hosts the patch file. http://www.logic.at/people/preining/iriver/patches.full/ i guess you can test it out on how it sorts die.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: lalittle on June 11, 2007, 04:39:11 PM
Can it be made to be dependent on the selected language?


That's an excellent point.  The patch should incorporate the ability to determine the language being used, and only ignore the articles in that language.

Thanks for specifically pointing that out.

Larry
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: Llorean on June 11, 2007, 04:48:08 PM
And why would someone only want to ignore articles in one language, but leave them in another, with multilingual tracks?

Is "The Who" "El Who" in Spanish, or still "The Who"? Meanwhile, "Los Pericos" (first band I could find starting with Los) should still be "Los Pericos" or should the Los no longer be ignored if the UI is in English? What about the song "Los Angeles", or should that name be used in the name of a band? Clearly that wouldn't be ignored.

Again, I don't see how this solution offers any amount of real flexibility.

Can you name some benefits other than "Users don't have to fix their tags" over the more general solution?
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: safetydan on June 11, 2007, 05:26:08 PM
As Llorean as made clear, a system that automagically determines which articles to drop is unlikely to be comitted to Rockbox. It's also unlikely to ever work well enough to satisfy everyone. Sorting on special tags will work for everyone since it's pretty much infinitely customisable.

So can I be a pain and ask people to test http://www.rockbox.org/tracker/task/7287 which is a patch that adds support for the sorting tags in ID3, Vorbis/APE comments, and MP4 metadata? It should add three new tags to the tagnavi syntax, sortalbum, sorttitle, and sortartist.

I've had a request to support an albumartist sort tag but that might be a bit messier as there's no separate tag in ID3 for that.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: alienbiker99 on June 11, 2007, 05:47:51 PM
i dont know if there is a way to do this, but setting up a system of checking which ones to ignore would work too
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: lalittle on June 11, 2007, 07:13:42 PM

And why would someone only want to ignore articles in one language, but leave them in another, with multilingual tracks?

Is "The Who" "El Who" in Spanish, or still "The Who"? Meanwhile, "Los Pericos" (first band I could find starting with Los) should still be "Los Pericos" or should the Los no longer be ignored if the UI is in English? What about the song "Los Angeles", or should that name be used in the name of a band? Clearly that wouldn't be ignored.


That's a valid point, but the short answer would be that I would have no problem with "Los Pericos" being listed under "L" rather than "P."  To me, the logic behind the ignore approach is to ignore articles in the currently selected language, and let everything else sort as it may.  I admit that the "ignore" approach will run into certain situations where the logic is not necessarily ideal to everybody, but I have found that in practice, the system works VERY effectively on devices like the iPod, as well as applications like iTunes, JR Media Center, and others.  These use the "ignore" approach to sorting without articles, and to put it simply, it works fine in my opinion.

Quote
Again, I don't see how this solution offers any amount of real flexibility.


I won't argue that the "tag" method can offer a lot more flexibility, but in my opinion it's not worth the cost (i.e. the extra work), which I believe is the reason that so few other applications or devices (no mainstream ones that I know of at this time) use the "ignore" approach.  To put it simply, it's just easier for the user.  Again, I understand that others don't necessarily agree with this -- I am only stating my opinion.

Quote
Can you name some benefits other than "Users don't have to fix their tags" over the more general solution?


No -- that's the only reason for using the "ignore" approach to this issue that I can think of.  To me, however, (and to other users like me who just don't want to have to deal with more tagging duties) this is a VERY important reason, and is a "deal breaker" for the "tag" approach.  You can call it "laziness" if you want, but the simple truth is that I'm already over-capacity on this sort of thing, and given that the "ignore" approach is already proven to work for me in my personal use, I prefer the concept.

Please note that I am not trying to change anybody's mind here -- I'm simply having a discussion where I'm clarifying my thoughts on the issue for anyone that is interested.

Thanks,

Larry
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: lalittle on June 11, 2007, 07:29:15 PM

As Llorean as made clear, a system that automagically determines which articles to drop is unlikely to be comitted to Rockbox.


I understand this and fully accept the situation.  This does not mean, however, that I won't continue to participate in discussions about the issue, or maintain hope that independent users will continue to develope a patch that uses the "ignore" approach for sorting.  Just because it's not officially committed doesn't mean that individuals can't create and apply the patch (which is one of the nice things about Rockbox in general.)

Quote
It's also unlikely to ever work well enough to satisfy everyone. Sorting on special tags will work for everyone since it's pretty much infinitely customisable.


I'm not sure why we keep coming back to this idea that the "tag" approach "will work for everybody."  As I've stated, it simply does not work for me because I'm not willing to go through the extra effort of editing tags for this purpose.  Again, you can call this laziness, but to me, it's more important to have it "automatic" (requiring NO user intervention) than it is to have it more "customizable."  I honestly don't NEED the extra customizing power of the "tag" approach, so the "ignore" approach, to me, is a the better option.  That's my opinion -- it's not "right" or "wrong," it's just my opinion.  Just because you don't agree with this does not invalidate the opinion.

On a side note, I'm clearly not the only one who feels this way, as evidenced by the use of the "ignore" approach in so many other devices and applications.  I'm not claiming that this proves it's the "correct" way to do this -- I'm simply pointing out that I'm not alone in my opinions here.

Thanks again for the discussion on this,

Larry
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: safetydan on June 11, 2007, 07:38:59 PM
If you want it to be automatic there are plenty of tools out there that will automatically populate your music files with the correct metadata. Heck, if someone was motivated enough they could write a plugin for Rockbox to go through all your music and copy track details to the sort tags while ignoring articles.

So if people still want something similar to the "ignore the" patch, perhaps they should work on making a plugin to do what I described above. That way we can have things the "right" way in the core, and the "wrong" way is isolated in a plugin.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: Llorean on June 11, 2007, 07:47:32 PM
And just to clarify, if it's not clear from above, what that would mean is that if you simply run that plugin, after you've run it, no other user steps, Rockbox would then act like it had the "Ignore The" patch for the database (or ignore articles based on whatever list of articles the plugin had) without any further user intervention (assuming the TSOP support was also implemented by that point). And a plugin like that could surely get accepted into SVN, meaning you wouldn't have to have the hassle of using an unsupported build and depending on someone else for your updates or maintaining the patch.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: hotwire on June 11, 2007, 09:23:45 PM
The problem as it appears in the English language, to me at least is as follows: usage of the definite article 'the' in artist/performer titles is ambiguous at best.

In examples such as 'The Beatles', 'The Who', 'The Music', clarity of artist identity is lost on deletion of, or rearrangement of the definite article, but alphabetizing of the a list of these alongside other items lacking a preceding definite article almost always results in a sorting that ignores said definite articles.  This is why for many English speaking Rockbox users, the necessity of an "ignore 'the'" patch is essential for his or her Rockbox experience.  The only examples I can think of offhand of a sorting that does not ignore 'the' are Rockbox, and the Windows file systems.  In fact isn't the official name for the artists above invoke the definite article, and thus any rename results in an inaccurate tag?

As for that ambiguous example: The Arcade Fire... or is it Arcade Fire?  Last.fm states the former, and the English Wikipedia states the latter, although going back a short number of revisions "The Arcade Fire" is used by Wikipedia also.  Their album covers use the latter, and yet all mentions of them in common speech (the same common speech that uses 'their' in place of 'his or her'), I only ever hear the band referred to as The Arcade Fire.  I am absolutely certain with a little effort, someone on this forum could even find two albums by an artist by which the artist uses a definite article in their name for one album, and does not in another.  Which one takes precedence? (this is intended more as a rhetorical question).

In the case of an album name or song title, usage of the definite article is not as ambiguous.  Take "The Black Parade", or "The Rising": both of these are formal titles to which usage of the definite article is deliberate for titling and thus must not be ignored when performing an alphabetized sort.  In other words, any 'ignore the' patch should only apply to artist/performer name and not to album or song title.

There you folks have it in a nutshell as to why in my opinion the requiring of users to perform a batch rename of tags to relocate any preceding definite article (henceforth referred to as "Artist, The") is in fact the inelegant shortcut for when a user wants to perform a sorting of artist names as is normally performed in English.  If you disagree, feel free to attempt to reorder any list on Wikipedia so that artist names starting with "The" appear under T, and see how long you last before your edit get reverted by one of those 'Wikipedian grammar freaks': http://en.wikipedia.org/wiki/List_of_best_selling_music_artists

I'm sure if someone has the will to dig around on Wikipedia for the policy on alphabetically sorting of lists we can find the proper procedure.  I bet it may even link to the norms in other languages.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: Llorean on June 11, 2007, 09:26:29 PM
Something tells me you completely ignored the whole conversation and chimed in at the end.

BOTH solutions put "The Beatles" between "Beast" and "Beautiful" if applied properly. Both show "The Beatles" when reading the list.

But one doesn't mandate that it goes there, doesn't mandate that other songs starting with "The" have their The ignored (completely flexible), and allows for articles in any language to be handled on a case by case basis.

The other doesn't.

English speaking Rockbox users aren't the only Rockbox users. A solution should encompass everyone, or be able to, and it should not have side effects that prevent its use. "Ignore The" breaks both of these concepts, as if used for English songs it could at the same time break non-English songs. And if non-English articles are added, it could break songs where the same string of characters isn't an article.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: hotwire on June 11, 2007, 09:31:03 PM
Sorry, my train of thought was considering the sorting order of current rockbox builds.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: Llorean on June 11, 2007, 09:33:43 PM
Current Rockbox builds sort strictly by exactly what is displayed.

The two offered solutions are

1) Ignore a pre-set list of articles, when enabled (possibly extended to enabling/disabling individual articles, but still impossible for a full multilingual solution)

2) Use a second set of standard (as in: part of existing standards) tags that contain a separate string, which is used for determining how it's sorted, while the normal string is what is displayed. For example, "The Beatles" would be the display string, but "Beatles, The" would be the sort string. The song would show up as "The Beatles" in the list, but its position would be determined by "Beatles, The". A plugin could be made to evaluate the songs on your player and create these tags if you didn't want to customize them on your own or have a PC-side utility that is capable.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: lalittle on June 12, 2007, 01:49:55 AM

And just to clarify, if it's not clear from above, what that would mean is that if you simply run that plugin, after you've run it, no other user steps, Rockbox would then act like it had the "Ignore The" patch for the database (or ignore articles based on whatever list of articles the plugin had) without any further user intervention (assuming the TSOP support was also implemented by that point). And a plugin like that could surely get accepted into SVN, meaning you wouldn't have to have the hassle of using an unsupported build and depending on someone else for your updates or maintaining the patch.


Based on the information at hand, this would get my vote.  I definitely agree that it would be really nice to have the feature officially supported by the developers, and eventually rolled into the builds by default rather than requiring patches and custom builds.  I assume it would take a little while for the "ignore" plugin to do it's thing and create the "advanced tags," but given that it only requires the user to say "do it" and then leave the iPod alone for a little while, that's fine.

On this note, am I correct in assuming that the process of creating new tags and re-writing the files could potentially take a considerable amount of time?  If so, perhaps 1) the "advance tag" plugin could revert to reading the standard name tags for sorting as long as the "sort" tags were blank, and 2) the "ignore" plugin could be make to ONLY process names that actually begin with an article and leave the rest alone.  This way, only the files that NEEDED the special sorting (i.e. files with articles) would be processed/rewritten, so the process wouldn't waste time re-writing files that didn't begin with articles.  Does this seem logical?

Thanks again,

Larry
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: Llorean on June 12, 2007, 10:31:13 AM
The plugin would have to read every tag, because it'd be impossible for it to know in advance which files have an article and which don't, but of course it'd only write the sort tags for the ones that needed changing, if done properly.

Also, as was said far, far earlier in this discussion, a very simple PC-Side program or script could do the exact same thing as this plugin, on a computer, as well automating the process and doing it in a far, far, far shorter time.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: Walrus on June 12, 2007, 10:51:27 AM


2) Use a second set of standard (as in: part of existing standards) tags that contain a separate string, which is used for determining how it's sorted, while the normal string is what is displayed. For example, "The Beatles" would be the display string, but "Beatles, The" would be the sort string. The song would show up as "The Beatles" in the list, but its position would be determined by "Beatles, The". A plugin could be made to evaluate the songs on your player and create these tags if you didn't want to customize them on your own or have a PC-side utility that is capable.



I still don't see why this is so complicated, (I mean, if it's implemented and someone doesn't like it, they can always turn it off) but...that option of having a separate 'sort' tag in the ID3 would work.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: Llorean on June 12, 2007, 10:56:48 AM
Features take up binary space. Saying "You may as well implement" is saying "you may as well hurt other people's experience for my benefit" because that binary space could be used for something else.

If a feature is implemented, it should be as small as possible, with the greatest overall benefit as possible. If this means a little bit more work for the end user to set up, but allows fully multilingual improved sorting rather than a improved sorting for a very limited subset, I think the "forget you, I don't want to have to do an extra step, the other people who get left out entirely with this feature if it's done this way can just bite me" argument is just a little bit unacceptable. Especially since the idea that "it doesn't cost the people who don't use it anything" is false. You don't have "Group A gains Sorting, Group B loses nothing." Group B gets a very, very little bit less RAM to store their music in. A little bit that counts on some players, and hardly matters at all on others, but it isn't free. So if the feature works, at the very least people of *any* language should have the option to use it freely and flexibly.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: lalittle on June 12, 2007, 02:30:19 PM

The plugin would have to read every tag, because it'd be impossible for it to know in advance which files have an article and which don't, but of course it'd only write the sort tags for the ones that needed changing, if done properly.


That seems logical to me.  It's the "re-writing" process that would take most of the time, so as long as it didn't have to re-write the files that didn't require it, it would save a great deal of time.

Quote
Also, as was said far, far earlier in this discussion, a very simple PC-Side program or script could do the exact same thing as this plugin, on a computer, as well automating the process and doing it in a far, far, far shorter time.


My preference for keeping the process within Rockbox is because for people like myself that use a music organization program that does not make use of the sort tags, Rockbox is the only thing that would utilize them.  I'd therefore prefer to not have a separate program manipulate the tags -- I'd like to keep the original files unchanged.  My personal preference, in other words, would be to keep the process entirely self-contained within Rockbox.  This way, anybody who preferred the "ignore" approach could achieve "sorting without articles" by navigating the Rockbox menu and selecting the "create article-free sort tags" option.  I think this would make the feature much more "accessible" to a much wider group of people.

Thanks again,

Larry
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: soap on June 12, 2007, 03:52:26 PM
Quote from: lalittle link=topic=10689.msg83587#msg83587

My preference for keeping the process within Rockbox is because for people like myself that use a music organization program that does not make use of the sort tags, Rockbox is the only thing that would utilize them.  I'd therefore prefer to not have a separate program manipulate the tags -- I'd like to keep the original files unchanged.

I don't see the separation you are seeing.  For either way this is implemented there will be a separate program manipulating the tags.  The only question is on what platform will said program run.

There is no difference between a Rockbox plugin creating sort tags and a "separate program" (PC side) manipulating the tags of the files residing on your DAP.  

A pluggin is nothing more than that, a "separate program", just one which runs on a small processor with limited resources.

There appears to be no difference between a PC side application and a DAP side application (pluggin) outside the aesthetic (and, again,  the limited resources affecting the DAP).  You must have your DAP plugged into the computer to transfer new files, might as well use the computer to create sort tags at that time.


Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: lalittle on June 12, 2007, 05:10:04 PM
I don't see the separation you are seeing.  For either way this is implemented there will be a separate program manipulating the tags.  The only question is on what platform will said program run.


In one case, an entirely separate program is required -- this program must be downloaded, installed, and configured to run a specific script.  In many cases, the re-written files will then have to be either updated or re-imported into the libary of the management program being used on this system.  Rockbox users would also need to have prior knowledge of this separate process that existed outside of Rockbox -- i.e. it wouldn't be inherently apparent that this capability existed.

In the other case, everything needed for the system to work would be contained on the Rockbox itself, and since it would be a menu item, it would be visible to everybody as an available feature.  Since the Rockbox sorting capability would be the only use for the advanced tags for many people (given that many mainstream programs don't utilize these tags) I feel it makes sense to completely contain this capability within Rockbox.  The feature would be right there in the Rockbox menus for all users to see, which would make it readily accessible to everybody who wanted to utilize this feature.  If a separate, outside process was needed for this, many people would never even know about it.

On top of this, I would rather not manipulate my original files any further if I don't have to, and I prefer to use a single application for this rather than one app for my "main" audio management duties, and another for this specific purpose.  In the past, there have been various "issues" related to editing tags with more than one program.  These issues are not that common, but they do exist.

Quote
There is no difference between a Rockbox plugin creating sort tags and a "separate program" (PC side) manipulating the tags of the files residing on your DAP.


There may be no difference to the end result, but there is definitely a distinct difference in work-flow philosophy.  This work-flow difference may not matter to some people, but it does to others.

Quote
A pluggin is nothing more than that, a "separate program", just one which runs on a small processor with limited resources.


But since this is a Rockbox specific feature to many (if not most) people, it makes sense (in my opinion) to offer the "complete" process from within Rockbox.  Again, this will make the feature more "accessible" to a wider group of people.  I honestly believe that if this feature was offered (i.e. the ability to have Rockbox add these "sort tags" itself via a self-contained process), a lot of people would take advantage of it.

Larry
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: soap on June 12, 2007, 05:28:30 PM

I don't see the separation you are seeing.  For either way this is implemented there will be a separate program manipulating the tags.  The only question is on what platform will said program run.


In one case, an entirely separate program is required -- this program must be downloaded, installed, and configured to run a specific script.  In many cases, the re-written files will then have to be either updated or re-imported into the libary of the management program being used on this system.  

No - you merely retag files while they reside on your DAP.  You keep stressing this point, but it is an invalid one.  There is no need for this process to interfere with your music management program of choice.

Quote
Rockbox users would also need to have prior knowledge of this separate process that existed outside of Rockbox -- i.e. it wouldn't be inherently apparent that this capability existed.

As opposed to Rockbox users needing to have prior knowledge of the plugin?

Quote

In the other case, everything needed for the system to work would be contained on the Rockbox itself, and since it would be a menu item, it would be visible to everybody as an available feature.  Since the Rockbox sorting capability would be the only use for the advanced tags for many people (given that many mainstream programs don't utilize these tags) I feel it makes sense to completely contain this capability within Rockbox.  
 Be it a plugin or a PC side application the end results would be the same - modified metadata on the audio files residing on your DAP.  Whether this affects your usage of "mainstream" music management programs or not is inconsequential, as said program will be equally affected either way.

Quote
The feature would be right there in the Rockbox menus for all users to see,

Not to split hairs, but it will be a pluggin, not a menu item.
Quote
which would make it readily accessible to everybody who wanted to utilize this feature.  If a separate, outside process was needed for this, many people would never even know about it.

Failure to read the manual affects people's knowledge and understanding of the "Random Folder Advance" pluggin.  How would this one be different?

Quote

On top of this, I would rather not manipulate my original files any further if I don't have to,
Again - nowhere did anyone suggest modifying your original files.
Quote
and I prefer to use a single application for this rather than one app for my "main" audio management duties, and another for this specific purpose.  In the past, there have been various "issues" related to editing tags with more than one program.  These issues are not that common, but they do exist.

Straw man - a broken tagger is a broken tagger.  A pluggin would be modifying your tags in the exact same fashion as a PC side app.

EDIT:spelling
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: safetydan on June 12, 2007, 05:42:54 PM
One thing that I think needs clarification is that support for sort tags (e.g. TSOP and company) is not a Rockbox specific feature. These tags are defined in the ID3v2.4 metadata specification and equivalents can be found in various Vorbis comment specs.

There are several other systems out there that can edit and make use of these sort tags. For example the Squeezebox from SlimDevices supports using these tags (well technically I think it's the SlimServer, but the point stands). So Rockbox is not alone in using these.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: Llorean on June 12, 2007, 05:45:49 PM
Yes, I pointed that it's part of several standards at the beginning of the explanation. The problem is, people seem to think that the software they choose to use is the only software that exists/matters. They also don't think spreading the use of a good standard is a good idea. :(

I think I'm going to state that from here on out, the argument for "Ignore The" that goes "It's easier / more convenient for my personal method of using it" is restricted. We get the point, we know you LIKE it better, but the idea has been rejected. Arguing for it accomplishes nothing, and since these forums are for discussing the state of official Rockbox, unless you have a new argument for it that you think will change the situation, you don't need to repeat what you've said in favour of it yet again.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: soap on June 12, 2007, 05:53:49 PM
I just want to clarify that I am not against using a pluggin as a solution to doing the tagging.  I was simply trying to dispel what I saw as an argument against a PC side application based upon faulty assumptions.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: lalittle on June 12, 2007, 07:49:43 PM
No - you merely retag files while they reside on your DAP.  You keep stressing this point, but it is an invalid one.


My point is not invalid -- there is simply a lack of clear communication on this point.  Editing these tags outside of Rockbox requires a program that can see and edit these tags, and it requires a script made specifically to do this.  This doesn't inherently exist with many of the mainstream audio management programs -- i.e. you can't necessarily do this in many of the most widely used applications used for this.  It would most likely require that a new, separate program be downloaded and installed, and a custom script run on the library.  After this, since the files had changed, it could likely require the audio management library to be manually "updated" in order to show the correct information for these files.

Quote
There is no need for this process to interfere with your music management program of choice.


Again, the mere fact that you'd likely need another, third party program is already a factor, not to mention the fact that re-writing the tags in different applications carries the "potential" (however unlikely) of extra problems.  These issues are completely eliminated if the actual unit is used to complete this task.  Also, given that at this time, for most users, Rockbox would be the only item that actually made use of these tags (iTunes, JR Media Center and other apps don't use them for their "ignore articles" sorting) it makes sense to put this functionality into Rockbox, allowing users to obtain article free sorting without the need of anything other than the Rockboxed unit itself.  This seems like a terrific feature that a lot of people who use iTunes or other mainstream apps would take advantage of.

Quote
As opposed to Rockbox users needing to have prior knowledge of the plugin?


The idea is that it would eventually be incorporated into the official build, which Llorean already indicated could be the case when he said "And a plugin like that could surely get accepted into SVN, meaning you wouldn't have to have the hassle of using an unsupported build and depending on someone else for your updates or maintaining the patch."  Once this was the case, it would appear in the menu, which would make it visible to anyone without requiring prior knowledge of it's existence.

Quote
Not to split hairs, but it will be a pluggin, not a menu item.


I don't understand -- all sorts of plugins have "menu items."  Take the iPod "Scroll Wheel Acceleration" for example.  This adds adjustments for this feature to the menus.

Quote
Failure to read the manual affects people's knowledge and understanding of the "Random Folder Advance" pluggin.  How would this one be different?


I don't follow you here.  My point was simply that with the myriad of features contained in Rockbox, having a menu item would alert people to the presence of this feature.  I support the idea of reading the manual, but I also know that this can be a daunting task for a complex item like this, so I also support the idea of making features more "visible," and therefor more "accessible" to the general user base.  This is simply the idea of making things more "user friendly."

Quote
Straw man - a broken tagger is a broken tagger.  A pluggin would be modifying your tags in the exact same fashion as a PC side app.


My point was that the original files that reside on the PC would be untouched.  Only the files on the Rockbox would be effected if the Rockbox itself took care of this duty.

Larry
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: Llorean on June 12, 2007, 07:54:35 PM
Scroll wheel acceleration is not a plugin.  Plugins are anything you seen in "browse plugins" or any viewers. Basically, files with the extension .rock. They do NOT show up in the main menus, with one exception (the credits plugin).

You seem to have confused it with the term "patch".

A PC-side program could very easily be run on the files on the DAP.

And I don't believe Rockbox should ever be forced to restrict or limit itself for fear of offending other programs that don't adhere to standards. "Fear of incompatibility" is silly, if that incompatibility is with a program that doesn't support a standard it claims to. If we add support for a standardized format, and another program can't deal with it that's supposed to also support that format, bug reports should be directed to them, rather than requests to us to remove (or in this case not add) or hack around that broken support.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: lalittle on June 12, 2007, 07:57:44 PM
I think I'm going to state that from here on out, the argument for "Ignore The" that goes "It's easier / more convenient for my personal method of using it" is restricted. We get the point, we know you LIKE it better, but the idea has been rejected. Arguing for it accomplishes nothing, and since these forums are for discussing the state of official Rockbox, unless you have a new argument for it that you think will change the situation, you don't need to repeat what you've said in favour of it yet again.


Wait -- did you think I was still arguing for the straight-out "ignore" approach?  If so, I've been very unclear.  What I've been trying to support in these recent posts is the idea originally suggested by safetydan above, where he suggested using the advanced tags for the "core" of the sorting function, but using a plugin to create these tags on the Rockbox itself.  This is what I said earlier "gets my vote," and what I've been attempting to support with these recent posts.

Thanks,

Larry
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: lalittle on June 12, 2007, 08:04:30 PM

Scroll wheel acceleration is not a plugin.  Plugins are anything you seen in "browse plugins" or any viewers. Basically, files with the extension .rock. They do NOT show up in the main menus, with one exception (the credits plugin).

You seem to have confused it with the term "patch".


Ah -- you're correct.  I was thinking of "plugins" and "patches" synonymously.  That said, a plugin would still reside on the Rockbox itself, and allow users to completely achieve article free sorting without needing anything else other than the actual Rockboxed unit.

Quote
And I don't believe Rockbox should ever be forced to restrict or limit itself for fear of offending other programs that don't adhere to standards. "Fear of incompatibility" is silly, if that incompatibility is with a program that doesn't support a standard it purports to.


It's not Rockbox I'd be worried about.  My concern would be potential complatibility issues between the other program used for editing these tags, and the user's audio management program of choice.  Again, these issues are 100% eliminated if the Rockbox itself took care of this task.

Larry
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: Llorean on June 12, 2007, 08:05:53 PM
If the user's audio management program supported the tagging specification properly, then where would the problem be exactly?

Not to mention, you don't have to run the program against your PC-side files. Just point it at the files on the device, click "go", and X minutes later the files on your device have their new, 100% legal according to specifications, tags. And much, much faster than it would take the device itself to update these tags by way of a plugin.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: soap on June 12, 2007, 08:37:14 PM

It's not Rockbox I'd be worried about.  My concern would be potential complatibility issues between the other program used for editing these tags, and the user's audio management program of choice.  Again, these issues are 100% eliminated if the Rockbox itself took care of this task.

No, these issues are not eliminated if Rockbox itself took care of said task.
A pluggin (in said example) is "the other program".
There would be no difference in the files be it a PC app or a Rockbox app (pluggin) that does the tagging.  The "risk" is exactly the same.
Quote
Editing these tags outside of Rockbox requires a program that can see and edit these tags, and it requires a script made specifically to do this.

Editing these tags inside of Rockbox requires a program (the plugin) that can see and edit these tags, and it requires a script something made specifically to do this.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: lalittle on June 13, 2007, 12:59:34 AM
No, these issues are not eliminated if Rockbox itself took care of said task.
A pluggin (in said example) is "the other program".
There would be no difference in the files be it a PC app or a Rockbox app (pluggin) that does the tagging.  The "risk" is exactly the same.


We're still not on the same page here.  The issues that I'm referring to are eliminated if the Rockbox takes care of this duty because the original files on the PC are untouched by the process.  Only the files already on the Rockbox are re-written, while the origian copies of these files on the computer remain exactly the same, completely "unaware" of the sort tags since these tags were not added until after the files were synced to the handheld.

Quote
Editing these tags inside of Rockbox requires a program (the plugin) that can see and edit these tags, and it requires a script something made specifically to do this.


Yes, but the distinction is that it happens within the Rockboxed unit.  No other external applications are necessary.  In other words, everything the Rockbox needs to achieve article-free sorting would be on the Rockbox itself.  No third party programs would need to be installed, configured, and utilized.  I'm simply advocating the idea that offering this "full service" capability on the Rockbox itself is an attractive feature to users like myself, and I believe that this would make the sorting feature more widely accessible.

Larry
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: GodEater on June 13, 2007, 01:40:02 AM

We're still not on the same page here.  The issues that I'm referring to are eliminated if the Rockbox takes care of this duty because the original files on the PC are untouched by the process.  Only the files already on the Rockbox are re-written, while the origian copies of these files on the computer remain exactly the same, completely "unaware" of the sort tags since these tags were not added until after the files were synced to the handheld.


Again, you're not getting it. If you plug your rockbox DAP into your PC - it becomes a drive letter. You point your tagging app at it, say tag these files here please. At no point then are your "original" files touched during the tagging process. At the end of the process, you unplug your DAP. You can even un-install the tagging app used. Your original files are still virgin, untouched on your PC's hard disk.


Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: lalittle on June 13, 2007, 03:29:54 AM
Again, you're not getting it. If you plug your rockbox DAP into your PC - it becomes a drive letter. You point your tagging app at it, say tag these files here please. At no point then are your "original" files touched during the tagging process. At the end of the process, you unplug your DAP. You can even un-install the tagging app used. Your original files are still virgin, untouched on your PC's hard disk.


AH!  You're right -- I was misunderstanding.   Now I get what you're saying.

Yes -- pointing the application at the handheld's drive would indeed eliminate this particular issue (as would having this done entirely "within" the Rockbox via a plugin.)

I guess this comes back to the general work-flow concept -- the idea of whether you do this entirely from within Rockbox, or with an external program.  Personally, I still prefer the idea of having the capability of doing the entire process on the Rockbox unit itself.  I like the idea of it being entirely self-contained, and not requiring another application.  To me, even if it takes longer to do it this way, it actually simplifies the process.

On a related note, would the re-writing of the files with the new tags cause any issues with the database, or does the database simply rely on the folder/name of the file?

Thanks again for the clarification, as well as the discussion in general.

Larry
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: Llorean on June 13, 2007, 01:07:06 PM
Both doing it in Rockbox or on with an external app would require you to re-initialize the database. It can't use the sort tags until it's included them in the database, and the database doesn't support updating tags on existing entries.
Title: observations from an interested party
Post by: jurrie on June 13, 2007, 02:47:29 PM
After having read the entire thread today I would like to add the following points/observations:



Please note that the plugin could interfere with sorting attributes previously associated with (provided by) the content.  The database's "update sorting attribute" API may wish to have a "force" parameter to override previous associations.  Similarly, one may envision a database setting which would prevent explicit sorting settings from being overwritten.

Edit: include cons
Title: Re: observations from an interested party
Post by: lalittle on June 13, 2007, 04:17:39 PM
I would argue that this MysteryPlugin should not alter the player's content in any way.  A more appropriate solution would appear to be that the MysteryPlugin would instruct Rockbox's database (via some new API) to update the sorting attributes associated with selected content.


Forgive me, but I'm unclear what this means.  How does it "update the sorting attributes"?  Could you clarify the details on what the process would involve?  I agree that it seems like a better idea to not alter the files, but I'm unclear how your suggestion would work.

Thanks,

larry
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: Llorean on June 13, 2007, 04:23:33 PM
I don't understand how it's a better idea not updating the files?

Better in what sense: Surely the files on your device aren't the primary copy of your music collection. That's generally a bad enough idea that you shouldn't be using an alternate firmware in the first place. Meanwhile, altering the files on the device allows the tags to be used by other programs (as there are programs that do support them), and insures that should you need to re-initialize the database you won't have to re-run the plugin.

The only advantage updating the database alone offers is "not having to change the files" which, in the end, isn't a very significant advantage. It just means that the information isn't available to any other part of Rockbox, or any tool that you use on your computer to keep your player in sync (or play your music, if you use your DAP as an external HD and play it with a PC tool to save battery in the case of those DAPs where this works).
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: lalittle on June 13, 2007, 05:11:39 PM

I don't understand how it's a better idea not updating the files?


My thinking was that the process could be significantly faster since it would not have to re-write the files, and the possibility of any sort of hiccups or corruption during this process would be eliminated.

Also, another "potential" concern of mine in all this has been how the re-writing of files might effect the "native" database on the unit.  I'm not sure about other units, but at least with iPods, Rockbox has always maintained a clean separation from the native database so that when booting to the native firmware, the iPod functions just like it would normally have.  This is important since there are still reasons to maintain full "native" functionality in certain situations.  Are there any potential issues with using the native firmware after re-writing the files via Rockbox?

Thanks,

Larry
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: Llorean on June 13, 2007, 05:53:23 PM
It would only make use of "legal" tags within the tagging specifications. This means the only time one could expect the original firmware to have problems is if it doesn't actually support the tagging specifications it claims to, in which case other "valid" files would break it just as easily.

All Rockbox would be doing is the same job many other taggers do. These tags aren't something we've made up, they're part of several specifications (though the only one for MP3 files that supports this, and is also supported by Rockbox, is ID3v2.4, it's also in Vorbis Comments, ApeV2, and I believe MP4 metadata though I could be wrong on the last one). In the case of the iPod for example, it would just mean more tags on the files that, because the Apple firmware doesn't support them, get ignored completely much like if you had additional comment tags or one of the other obscure tags that are part of the specification.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: ell1ps1s on June 13, 2007, 06:30:06 PM
Since version 7.1, iTunes does support sorting tags. As far as I can tell, this is not just within its own database; it does also write  TSOP etc. tags to the audio files. (At least on my Mac; I can't comment on the Windows version, although I would hope they behave in the same way.) It also seems to supplement its own behaviour of ignoring (English) articles by adding sort tags, which would thus be available to Rockbox if support for those tags were implemented.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: Llorean on June 13, 2007, 06:44:20 PM
I need to apologize, apparently there's no "official" way of doing it in Vorbis Comments. Which means I need to figure out where I "learned" that there was.

Either way, it looks like there's at least one mainstream program doing it this way now. And apparently, Apple has received complaints because "The Beatles" are sorted differently in the German version of iTunes than the English version because of differences in their list of articles... ;)
Title: Re: observations from an interested party
Post by: jurrie on June 13, 2007, 07:18:05 PM

Forgive me, but I'm unclear what this means.  How does it "update the sorting attributes"?  Could you clarify the details on what the process would involve?  I agree that it seems like a better idea to not alter the files, but I'm unclear how your suggestion would work.


I had envisioned that the plugin would have logic similar to:
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: jurrie on June 13, 2007, 07:38:08 PM

I don't understand how it's a better idea not updating the files?

First: it's a lot of content to copy for each file just to add one, or a few, tags.  Second (and this is admittedly personal preference): I prefer that a "Player" not modify content.

Quote

Surely the files on your device aren't the primary copy of your music collection.

Correct.   However, the primary copy is where the change should be made (a flac file, in my case) and then propogated/published downstream.

Quote

Meanwhile, altering the files on the device allows the tags to be used by other programs (as there are programs that do support them),

Programs inquiring about audio content could(should?) query the database instead.

Quote

 and insures that should you need to re-initialize the database you won't have to re-run the plugin.

Yup, that's one thing that I overlooked previously.  I've updated my previous post to reflect this "con".

In the end, it may just be a difference in philosophy.  I look at at Rockboxed players as players.  Content (including tags) should be created/manipulated elsewhere.  So.... if the perception is that this really is a "Rockbox issue" (which I don't - it's a tag accuracy/integrity issue), then having a sorting-tag fixer utility provided by the Rockbox community (a la, Rockbox Utility) would be my preferred solution.  Such a utility could be run on the primary audio content copy, or if the player's content is mounted on the utility's host it could be run on the Rockbox audio content copy.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: Llorean on June 13, 2007, 07:45:58 PM
The expectation is that the user will have files they've manually added these tags to.

The plugin would exist as a *means* of adding these tags should their current audio management tools not support the tags, or because they don't want to manually add them and wish to use a basic algorithm stripping a certain list of articles. This plugin should of course be a "last resort" as a tool doing this would work best on a computer where it could run faster.

The idea isn't that this plugin is a primary solution. Rather it's a last resort solution for those who absolutely refuse to tag their files manually or have a utility on their computer do it, and insist everything must be done on the player.

I don't think it would be a good idea to address the database directly, nor do I think plugins should be reading from the database for information about a song unless the plugin is directly related to the database because when doing so the database may not exist. Not that the sort tags are likely to be used outside the context of the database, but I think they're an attribute of the file metadata and should be kept in the file metadata.

Manipulating the database directly, a la iTunes, results in files that aren't properly usable elsewhere because some or all of the metadata a user expects to have been added to them by their actions doesn't show up. (iTunes has been known to not write tags to files, only updating the iPod database, making those files impossible to use in Rockbox since they have no recognizable filename nor complete metadata)
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: jurrie on June 13, 2007, 08:25:23 PM

The expectation is that the user will have files they've manually added these tags to.

Agreed.

Quote

The idea isn't that this plugin is a primary solution. Rather it's a last resort solution for those who absolutely refuse to tag their files manually or have a utility on their computer do it, and insist everything must be done on the player.

Right.... I guess we just differ in opinion on where this tool should run.

Quote

I don't think it would be a good idea to address the database directly,

Agreed.  That's why I suggested a database API.  The only information that would be available to be read, or changed, would be that which the database elected to make available.

I'll stop harping on this now.  The easiest way for me to contribute in this area is to provide the simple conversion tools for one or more common formats.  Guess I'll get with safetydan to see what format he's looking for.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: safetydan on June 13, 2007, 08:34:40 PM
I'll just chime in here and say that I've no interest in working on a tag editing plugin. Especially since even such mainstream applications as iTunes already support editing and maintaining the sort tags.

The only thing I'm going to address is getting the database to use the sort metadata (when it's available) for sorting album/artist/title for display.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: alienbiker99 on August 01, 2007, 10:31:54 AM
not to bring up this topic again, but i had a new idea. i was using mediamonkey and found out that it can ignore what ever words you want to ignore by typing it in one of the options. so i was thinking, why can't this kind of option be implemented into rockbox. this would solve the problem of not being multi-language. you would type it into an option which would create a .cfg then the database would ignore the word(s) that are in the config file + one space.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: Llorean on August 01, 2007, 11:53:42 AM
Because it doesn't handle the cases where a word needs to be ignored for one song (because it's in one language) but not ignored for another (because it's in a different language).
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: alienbiker99 on August 01, 2007, 11:58:36 AM
is there really a problem of languages overlapping their articles? and it would only ignore the leading article + one space
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: Llorean on August 01, 2007, 12:00:31 PM
Why does "it would only ignore the leading article + one space" matter, exactly?

And yes, we came up with some situations in the IRC channel where different languages had conflicting articles.

But the most important question is: How does this offer anything that the sort tags don't?
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: alienbiker99 on August 01, 2007, 12:07:08 PM
it only means that the leading article would be ignored. i don't know how its different, i just thought of it last night. it seems easier than the tags, but it might not be if there is conflicting articles
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: Llorean on August 01, 2007, 01:17:21 PM
Previous suggestions included ignoring the leading article, so what I meant was, how do you mean for that to be different from other things said?

As well, sort tags offer the ability to have names sorted differently, numbers such as the word "Three" treated the same as the number "3" for people who prefer that, etc. A vast amount of increased flexibility, and no language restrictions.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: soap on August 01, 2007, 04:11:26 PM

As well, sort tags offer the ability to have names sorted differently, numbers such as the word "Three" treated the same as the number "3" for people who prefer that, etc. A vast amount of increased flexibility, and no language restrictions.

Including bands (A Silver Mt. Zion I'm looking at you) who change their name every single album.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: ryran on August 01, 2007, 05:06:31 PM
Including bands (A Silver Mt. Zion I'm looking at you) who change their name every single album.
lol... bastards!
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: seani on August 02, 2007, 01:39:07 AM


Including bands (A Silver Mt. Zion I'm looking at you) who change their name every single album.

lol... bastards!


"bastards, the"

if you don't mind
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: Toxikator on September 20, 2007, 05:17:30 PM
Poor etiquette to bump an old thread, I know, but I was wondering if any progress had been made on the front of getting "sortartist" and "artist" to automatically work together in the database? (ie it sorts by one and displays the other, rather than recognizing but displaying the sorttag)

the flyspray seems to have not been updated, just curious...
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thr
Post by: safetydan on September 20, 2007, 06:01:55 PM
I've had no time recently to work on this patch (or anything else Rockbox related).  Unfortunately I don't see this changing for a while. Maybe try and convince Slasheri to have a go at it since he's the tagcache guy.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: Toxikator on September 21, 2007, 07:43:07 AM
Thanks, I don't want to bug anyone about it but I appreciate the heads-up.
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: Pricey on September 20, 2008, 10:33:48 PM
You guys are talking a language I don't really understand. I'm not really a coder I don't know the lingo.

Whats the outcome. Is the hiding of the "the" going to happen or has it been reged by RockBox gods?
Title: Re: Rejection of "ignore the" patch (split from the Evil_G unsupported build thread)
Post by: soap on September 20, 2008, 11:04:07 PM
Whats the outcome. Is the hiding of the "the" going to happen or has it been reged by RockBox gods?
It will happen when someone with the desire, the skills, and the time does it.

see above:
I've had no time recently to work on this patch (or anything else Rockbox related).  Unfortunately I don't see this changing for a while. Maybe try and convince Slasheri to have a go at it since he's the tagcache guy.