Offical Battery Life

[Eli Gundry]

When I look at the battery bench page, I really can't get an idea of how long the battery life is. I mean, when you look at Apple's iPod page, they come right out and tell you an amount of time its expected to last, under normal use. As of right now, Rockbox's battery lives are all over the place.

This is what I propose. We have two different pages. One for "Unofficial Battery Benches", which would be the page we have right now. The other would be for "Official Battery Benches", in which each bench follows exactly the same conditions as the others. These conditions would be things like the same playlist/album on repeat, the same WPS, same .config files, same type of headphones at the same volume, etc. And in order to account for discrepancies in hardware, we take several benches and just average them together.

Here are just a few guidelines I'd like to throw out here right now:

-Have Pink Floyd's "Dark Side Of The Moon" on repeat. It's one of the best selling albums ever and many audio professionals use it to test audio equipment.
-Have it encoded in LAME MP3 at 144 kbps CBR. Its between 128 and 192, which I'd imagine is the average bit rate of mp3s.
-Use the default Cabbie 2.0 WPS.
-The volume would be at -15 db. That just seems average to me.
-Album art turned on.
-I'm not sure about if most people turn hold on, but have it on.
-LCD powers off after 5 seconds.

I just want these official benches to mirror a normal users configuration, so if you have and ideas, toss them out there.

[Llorean]

Do you have a proposal for how we get a factory new player of each sort with a fresh battery on which to perform these tests?

Battery benches aren't "realistically" useful unless they meet your own use conditions anyway. So trying to match the Apple ones (which are actually 'best case' rather than 'realistic') doesn't seem to do much good.

What good would it actually do users?

[Eli Gundry]

As to the first part, you take an average.

To the second part, look at the current battery bench page. All I want to do is kind of eliminate the confusion that comes when you see an iPod Video's run time range from 5 to 20 hours. By doing it my way, you have a constant method of testing pretty much all platforms. It's supposed to sway new users. Most (normal) people will be confused if they see that mess of a runtime page.

And yeah, I suppose Apple's runtime's are best case also, but aren't you supposed to be better than them?

[froggyman]

As to the first part, you take an average.

To the second part, look at the current battery bench page. All I want to do is kind of eliminate the confusion that comes when you see an iPod Video's run time range from 5 to 20 hours.

well that happens because of the players being older and some being newer and some being less used and ect. you just can even them out, you just cant unless you were to use all new batteries

[Eli Gundry]

well that happens because of the players being older and some being newer and some being less used and ect. you just can even them out, you just cant unless you were to use all new batteries

The ones with 5 hours to 20 hours are outliers, as they either have old batteries or are using unrealistic settings for their benches. All I want is an average battery life for all players with unified test settings. By taking an average, you can avoid the problems of accounting for old or new batteries.

[Llorean]

Or the 5 hour one comes from a much older build than the 20 hour one, and the code has changed significantly since then.

Taking an average of just a few skews perception even worse unless you have an extremely large set of players because it suggests to people that the numbers are "good" when the sample set is too small to really determine that.. Or did you have an idea where we could get about a dozen players of varied use amounts for each type of player we support, where the average amount of wear for each set of a dozen is very close to that of each other set and is a describable amount of wear so that we can say to people "For people who've used their player X amount of time, this level of battery life is typical."

Taking an average reduces the outliers, but you need a large enough sample set for it to matter. If we average a dozen heavily used players (as is likely the case if we average people using Rockbox) it doesn't adequately represent runtime.

The *only* useful piece of information on that page is the comparison, on the same player, of the original firmware's runtime vs Rockbox's runtime with the same playlist. Even that is only useful if there is the possibility to match volumes (which there really isn't, at best you can only approximate).

Follow that by the fact that runtime changes over time as Rockbox changes, we'd have to be updating the page monthly, which would mean tracking down another set of players (hopefully with similar averages to the original sets) and perform the series of tests every month. And then come up with a way to adjust for the fact that the batteries are aging unless we wanted to also do runtime tests in the original firmware each time, which is difficult at best to get accurately since there's no logging apparatus so you cant just check the voltages over time like you can with Rockbox.

Now, mind you, if you feel like volunteering to coordinate such an effort monthly, nobody's going to stop you. But getting numbers that are "accurate" is really impossible. The most meaningful you can get is "under similar conditions, Rockbox tends to match or surpass the original firmware's runtime." We can already comfortably say this.

If you'd like a number that *is* meaningful, come up with a way to hook some hardware up and measure the average wattage across several hours, so you can tell people how much power Rockbox draws compared to the original firmware. This would only require one player of each type, does not require running through a full charge (or several, to average out for each individual player and avoid a bad measurement) and should not be affected by device/battery age.

[saratoga]

The ones with 5 hours to 20 hours are outliers, as they either have old batteries or are using unrealistic settings for their benches. All I want is an average battery life for all players with unified test settings. By taking an average, you can avoid the problems of accounting for old or new batteries.

This really makes no sense at all.  Aging isn't a zero mean process.  You can't average it out.  All an average would do is weight points along a capacity vs. age curve by the number of users at each point.  The result would be more or less meaningless.

If you just want a power consumption measurement independent of the battery, get a DMM and measure current.

[Eli Gundry]

Alright, I guess that makes sense. Oh well, seemed like a good idea.

[psycho_maniac]

this makes me want to do another battery bench but i been using my player a lot lately and this is a different player (same one just newer) and this one seems to last a lot longer then my old one.

[soap]

I think the point is being missed that a battery bench at any point during the battery's lifetime can be quite meaningful IF said battery bench is "calibrated" by performing a baseline test on the Original Firmware.

As to the idea of using "Dark Side of the Moon"... There are plenty of free albums on archive.org which would be available to all.  Not to mention a common source and common encode removes yet more variables from the equation.

This is something which could totally be done as a distributed project.  All that needs done in preparation is documenting a standard testing procedure - the specifics don't matter as long as you're consistent, and a standard set of encodes.  Said standard should include specifics for each and every target's OF (where possible) and a standard .config file for each Rockbox target.

Then, and only then, could a statement such as "Using 'X' testing procedurem Rockbox achieves N% of Original Firmware runtime on target Y."
Such a statement would be true regardless of battery condition or hardware modifications (HDD size, CF card, mega battery, etc.)
Such a statement would also be quite useful, and easy to track over time.

 

[Llorean]

That'd still be pretty inaccurate. Because spinup times happen at such different points it's not at all unimaginable that a battery in one state would show Rockbox worse, and a battery in another state shows Rockbox better, due to the final spinup happening in slightly different times in the playlist for each Firmware (if Rockbox's battery lifetime were within ~30 minutes - 1 hour of the OF's on that player).

On flash player's it's different, but hard disk spinups can very much skew a set of results, especially if you're ~1 or 2% from full rather than being perfectly full in the OF (which rarely shows you accurate information beyond an image of a battery).

Since we don't have an OF battery bench, all we can do is start music, and record when it ends. You don't even have a log of voltages so you can't approximate how long it took from 90% to 10% or anything like that.

Basically, you could have an organized procedure but there's no way to ensure getting consistent output from the OF, so at best it's still a very, very loose estimate.