Rockbox.org home
Downloads
Release release
Dev builds dev builds
Extras extras
themes themes
Documentation
Manual manual
Wiki wiki
Device Status device status
Support
Forums forums
Mailing lists mailing lists
IRC IRC
Development
Bugs bugs
Patches patches
Dev Guide dev guide
Search



Donate

Rockbox Technical Forums


Login with username, password and session length
Home Help Search Staff List Login Register
News:

Thank You for your continued support and contributions!

+  Rockbox Technical Forums
|-+  Support and General Use
| |-+  Audio Playback, Database and Playlists
| | |-+  Codec Efficiency Comparison Test (iPod)
« previous next »
  • Print
Pages: 1 [2] 3 4

Author Topic: Codec Efficiency Comparison Test (iPod)  (Read 37570 times)

Offline Davide-NYC

  • Member
  • *
  • Posts: 429
Re: Codec Efficiency Comparison Test (iPod)
« Reply #15 on: October 02, 2006, 06:49:13 PM »
If I'm testing 21 files I want to make sure I'm testing the file I'm supposed to be testing.
I know myself and I know I'll get distracted or miss a file or whatever. This way confusion is impossible.  I also want to minimize the WPS as much as possible in terms of CPU load. If we're to solicit the regular users (like me) in helping us test I feel it necessary to give something clear and comprehensible to look at.

Don't worry, I don't have CVS write access and I don't want it.
Logged
Currently: iRiver H132-RTC-CFMod

Offline Rincewind

  • Member
  • *
  • Posts: 266
Re: Codec Efficiency Comparison Test (iPod)
« Reply #16 on: October 03, 2006, 05:09:30 PM »
maybe a nice solution would be a real (unofficial, nothing in cvs of course) testing version of rockbox with dedicated testing features like flushing the audio thread, clearing buffers with button presses, triggered timers...

but this would increase cpu load a little bit. But the question is: do we want to find out what the settings with the best cpu efficiency are or do we want to find the "best" codec?
Logged
Iriver H120, Sansa e280

Offline senab

  • Artist
  • Member
  • *
  • Posts: 188
  • The Mighty Senab!!!
    • senab.co.uk
Re: Codec Efficiency Comparison Test (iPod)
« Reply #17 on: October 05, 2006, 03:49:35 AM »
How do you define the best decoder then?

Some people want more battery life
Some people want to fit more music on their player
Some people want both

What's best is personal to the user  ;)
« Last Edit: October 05, 2006, 02:22:44 PM by senab »
Logged

Offline Davide-NYC

  • Member
  • *
  • Posts: 429
Re: Codec Efficiency Comparison Test (iPod)
« Reply #18 on: October 05, 2006, 10:52:42 AM »
My intention was just to see how different codecs performed across targets at a given time.

(been busy lately, will get to the wiki page ASAP)
Logged
Currently: iRiver H132-RTC-CFMod

Offline soap

  • Member
  • *
  • Posts: 1678
  • Creature of habit.
Re: Codec Efficiency Comparison Test (iPod)
« Reply #19 on: October 07, 2006, 07:08:08 PM »
The wiki entry is looking nice, but after looking at Senab's encode.bat I feel it is important to say that these are not CBR encodings - despite the implications made by both the filenames and the wiki page table.

 
Logged
Rockbox Forum Guidelines
The Rockbox Manual
How to Ask Questions the Smart Way

Offline Davide-NYC

  • Member
  • *
  • Posts: 429
Re: Codec Efficiency Comparison Test (iPod)
« Reply #20 on: October 07, 2006, 07:17:14 PM »
OK here is the (very preliminary) wiki page. http://www.rockbox.org/twiki/bin/view/Main/CodecPerformanceComparison

I fixed senab's batchfile (it had a tiny bug in it) and I modified the patch to disply filename, codec, and bitrate in the 'view audio thread' screen.

Let me know what you all think. Running tests now.
Logged
Currently: iRiver H132-RTC-CFMod

Offline saratoga

  • Developer
  • Member
  • *
  • Posts: 8987
Re: Codec Efficiency Comparison Test (iPod)
« Reply #21 on: October 07, 2006, 07:58:16 PM »
Quote from: Davide-NYC on October 07, 2006, 07:17:14 PM
OK here is the (very preliminary) wiki page. http://www.rockbox.org/twiki/bin/view/Main/CodecPerformanceComparison

I fixed senab's batchfile (it had a tiny bug in it) and I modified the patch to disply filename, codec, and bitrate in the 'view audio thread' screen.

Let me know what you all think. Running tests now.


That looks fantastic.
Logged

Offline soap

  • Member
  • *
  • Posts: 1678
  • Creature of habit.
Re: Codec Efficiency Comparison Test (iPod)
« Reply #22 on: October 07, 2006, 08:39:16 PM »
Regarding again the target bitrates:
What is the bitrate variation amongst the encoded files?  Are the mp3 encoded files within reasonable tolerances of their ogg counterparts?
The reason I ask is if the "128kb/s" MP3 is actually 120, and the "128kb/s" OGG is actually 132 (or vise versa) that would be more than enough to call the results into question.
Logged
Rockbox Forum Guidelines
The Rockbox Manual
How to Ask Questions the Smart Way

Offline Davide-NYC

  • Member
  • *
  • Posts: 429
Re: Codec Efficiency Comparison Test (iPod)
« Reply #23 on: October 08, 2006, 12:38:33 PM »
Quote from: soap on October 07, 2006, 08:39:16 PM
The reason I ask is if the "128kb/s" MP3 is actually 120, and the "128kb/s" OGG is actually 132 (or vise versa) that would be more than enough to call the results into question.

Hmm. Agreed, I'll modify the batch file to encode in CBR in all possible instances.

This morning I was trying to test with an iRiver H3x0 and realized that since there is no HD activity LED on the unit a quick way to determine whether or not there was any disk activity during the boost ratio calculation would be to enable the status bar in the "view audio thread" screen. Good idea?

Of course I have no idea how to do this codewise.  ;D

Any care to chime in? I would modify the VAT screen patch accordingly.
Logged
Currently: iRiver H132-RTC-CFMod

Offline bk

  • Member
  • *
  • Posts: 266
Re: Codec Efficiency Comparison Test (iPod)
« Reply #24 on: October 08, 2006, 01:05:53 PM »
Quote from: Davide-NYC on October 08, 2006, 12:38:33 PM
Hmm. Agreed, I'll modify the batch file to encode in CBR in all possible instances.

Forcing CBR for natively VBR formats (Ogg Vorbis) would be misleading. I think the value of these benchmarks is not necessarily in the comparison but rather in the absolute boost level per target. In other words it's less useful to compare codec A to codec B, but more useful to know that codec A is 20% slower on ARM relative to ColdFire.
Logged

Offline soap

  • Member
  • *
  • Posts: 1678
  • Creature of habit.
Re: Codec Efficiency Comparison Test (iPod)
« Reply #25 on: October 08, 2006, 02:21:26 PM »
Quote from: bk on October 08, 2006, 01:05:53 PM
Quote from: Davide-NYC on October 08, 2006, 12:38:33 PM
Hmm. Agreed, I'll modify the batch file to encode in CBR in all possible instances.

Forcing CBR for natively VBR formats (Ogg Vorbis) would be misleading. I think the value of these benchmarks is not necessarily in the comparison but rather in the absolute boost level per target. In other words it's less useful to compare codec A to codec B, but more useful to know that codec A is 20% slower on ARM relative to ColdFire.

Forcing at least ABR is a must for this test to mean anything.  Comparing MP3 to OGG when the bitrates differ substantially is worthless.

As for the original encodings, only 192 and 256 are useful, IMHO.
Logged
Rockbox Forum Guidelines
The Rockbox Manual
How to Ask Questions the Smart Way

Offline senab

  • Artist
  • Member
  • *
  • Posts: 188
  • The Mighty Senab!!!
    • senab.co.uk
Re: Codec Efficiency Comparison Test (iPod)
« Reply #26 on: October 08, 2006, 04:39:07 PM »
Yes this test wasn't really a test on bitrates against each other and you couldn't really do such a test. Bitrates will change on the individual sample, (ie Vorbis q6 is supposedly ~192, but gives me ~180 very often).

What you can do with the results is plot a bitrate vs boost ratio scatter graph and then create a line of best fit to compare how the bitrates between teh codec's.
Logged

Offline soap

  • Member
  • *
  • Posts: 1678
  • Creature of habit.
Re: Codec Efficiency Comparison Test (iPod)
« Reply #27 on: October 08, 2006, 04:55:46 PM »
Quote from: senab on October 08, 2006, 04:39:07 PM
Yes this test wasn't really a test on bitrates against each other and you couldn't really do such a test. Bitrates will change on the individual sample, (ie Vorbis q6 is supposedly ~192, but gives me ~180 very often).

What you can do with the results is plot a bitrate vs boost ratio scatter graph and then create a line of best fit to compare how the bitrates between teh codec's.

Why couldn't you force ABR or CBR on the codecs that support it?  Even if the different codecs allocate bits inconsistently throughout the song, the test is the average boost percentage for the song is it not?  So as long as the average bitrates are comparable the test should provide a valid comparison.
« Last Edit: October 08, 2006, 05:05:19 PM by soap »
Logged
Rockbox Forum Guidelines
The Rockbox Manual
How to Ask Questions the Smart Way

Offline senab

  • Artist
  • Member
  • *
  • Posts: 188
  • The Mighty Senab!!!
    • senab.co.uk
Re: Codec Efficiency Comparison Test (iPod)
« Reply #28 on: October 08, 2006, 04:59:11 PM »
Well thats it, would using CBR put less of a strain on the CPU? It would be an unfair test then. Here's the type of graph I was thinking of:

Logged

Offline soap

  • Member
  • *
  • Posts: 1678
  • Creature of habit.
Re: Codec Efficiency Comparison Test (iPod)
« Reply #29 on: October 08, 2006, 05:10:19 PM »
Maybe CBR would present less of a strain, but I believe most/all support ABR.

Regardless, your graph has won me over.
EDIT:
(as long as the X-axis is the actually achieved ABR, not the attempted BR)
« Last Edit: October 08, 2006, 05:12:05 PM by soap »
Logged
Rockbox Forum Guidelines
The Rockbox Manual
How to Ask Questions the Smart Way

  • Print
Pages: 1 [2] 3 4
« previous next »
+  Rockbox Technical Forums
|-+  Support and General Use
| |-+  Audio Playback, Database and Playlists
| | |-+  Codec Efficiency Comparison Test (iPod)
 

  • SMF 2.0.17 | SMF © 2019, Simple Machines
  • Rockbox Privacy Policy
  • XHTML
  • RSS
  • WAP2

Page created in 0.155 seconds with 22 queries.