Support and General Use > Audio Playback, Database and Playlists
Codec Efficiency Comparison Test (iPod)
Llorean:
As far as I'm concerned testing boost only gives you relevant information over the normal playback condition, and comparing it boost between codecs on the same platform is fine, but between platforms destroys the value of the data. If all the codecs require more boosting on one system than the other, is it that all the codecs are still poorly optimized, or is it that the playback code requires effort?
Knowing that an MP3 decodes at 300% speed on ARM7 @ 75mhz, and 350% speed on M68K @ 124mhz is much more relevant information to overall codec efficiency on those platforms.
If this were a Playback Efficiency thread then I would gladly admit that operating system concerns would be relevant to the topic (and they are relevant to users) but either way they should still be measured entirely seperately of the codec (playback system should probably be measured with WAV files).
Davide-NYC:
In trying to refine the testing process:
CBR, ABR or VBR? I think whatever we choose should be the most "default" coding option. I.E. the encoding options used the majority of the time. What is the batchfile going to tell the encoders to do exactly? Do we have concensus on this? My vote is use the encoder defaults because presumably that's what most file are encoded with.
OK, so the Coldfire and ARM data sets should not be merged. Do we need to separate the different PortalPlayer chips? PP5002, PP5020, etc?
Coldfire Set:
iriver iHP-100
iriver H120
iriver H140
iriver H320
iriver H340
iAudio X5
ARM Set:
iriver H10 5/6GB
iriver H10 20GB
iPod 3G
iPod 4G
iPod mini (1G & 2G)
iPod Color
iPod Nano
iPod Video
??? What about the statusbar in "VAT" screen for HD activity? Many targets do not have an HD activity LED. It's nice to be able to see if disk access is interfering with the CPU boost ratio. Please post some code for me so that I may expand the debug_menu.c patch further.
About the transcoder to WAV plugin:
This is a great idea. The plugin could output performance data to a logfile. Just like the battery_bench plugin. This would be MUCH EASIER for the user to do.
Once the Codec_Bench plugin is written (hint hint) this entire thread becomes moot right? We would just tell users to download and encode the test files and run the plugin until it is finished. Right?
Llorean:
A more useful feature would be a transcoding interface that just happens to have a "benchmark" option somewhere that causes it not to output a file. ;)
For your list of players, the primary things that will make a difference are: Screen and Processor. The available RAM should be entirely irrelevant to an actual codec test as long as you're making sure the whole file gets buffered, so the H110, H115 and H120 should perform identically to the H140, and the same is true between 320 and 340. The iPods are all different though because of either different screens or different PortalPlayer chipsets.
Davide-NYC:
--- Quote from: Llorean on October 09, 2006, 01:45:48 PM ---A more useful feature would be a transcoding interface that just happens to have a "benchmark" option somewhere that causes it not to output a file.
--- End quote ---
What I meant by output was just a logfile that listed decoding times and whatever else was relevant to the test...
Example:
Target and Build Date.
flac_5.flac --> time.
flac_8.flac --> time.
lame_096.mp3 --> time.
lame_128.mp3 --> time.
lame_192.mp3 --> time.
lame_256.mp3 --> time.
lame_320.mp3 --> time.
mpc_096.mpc --> time.
mpc_128.mpc --> time.
mpc_170.mpc --> time.
mpc_224.mpc --> time.
mpc_300.mpc --> time.
mpc_350.mpc --> time.
nero_096.m4a --> time.
nero_128.m4a --> time.
nero_192.m4a --> time.
nero_256.m4a --> time.
nero_320.m4a --> time.
nero_400.m4a --> time.
vorbis_096.ogg--> time.
vorbis_128.ogg--> time.
vorbis_192.ogg--> time.
vorbis_256.ogg--> time.
vorbis_350.ogg--> time.
vorbis_500.ogg--> time.
wv_fastx3.wv --> time.
wv_normx4.wv --> time.
That would rule and take all of the human error out. It would also be easy for the users to do.
Although, now that I think about it a transcoder that does "non-realtime-format-->WAV" would be a great tool for playing back files that are not playable on Rockbox (yet) while not in front of a computer. Hmmm! :)
Llorean:
No, I understand what you meant by output. Rather, what I meant was that instead of a codec benchmark plugin, it'd be more useful to spend the time working on a full transcoding interface, so you can take *any* supported file, and have it decoded to WAV and then encoded to the format of your choice.
Then, once that exists, a benchmark option could be added to it that fits most of the needs above, and as a bonus because it's a full transcode interface, could benchmark both encoding and decoding codecs. :)
By the "instead of outputting" bit, I just meant that when using the interface to benchmark, instead of saving a file it could log it or output time results to the screen.
Navigation
[0] Message Index
[#] Next page
[*] Previous page
Go to full version