Rockbox Technical Forums

Support and General Use => Audio Playback, Database and Playlists => Topic started by: Febs on August 26, 2006, 09:47:38 PM

Title: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: Febs on August 26, 2006, 09:47:38 PM
I used Rightmark Audio Analyzer to compare Rockbox versus the original software.

The results are set forth in successive posts below.

All tests were performed on my 2.8gHz Pentium 4 PC with an Audigy 2 sound card.  Other test conditions are noted in the individual posts below.



Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: Febs on August 26, 2006, 09:51:37 PM
Rockbox versus original Apple firmware.
iPod video 60GB
Headphone output
Volume at 0dB in Rockbox, matched as closely as possible in the Apple FW using RMAA's level check.

Rockbox frequency response (headphone output):

(http://jazzexplosion.com/Febs/Rockbox/rockbox_freq_res.png)

Rockbox frequency response (line output using Pocket Dock):

(http://jazzexplosion.com/Febs/Rockbox/ipod_rockbox_lineout_freqres.png)


Apple firmware frequency response:

(http://jazzexplosion.com/Febs/Rockbox/apple_freq_res.png)

Rockbox dynamic range:

(http://jazzexplosion.com/Febs/Rockbox/rb_dynamic_range.png)

Apple FW dynamic range:

(http://jazzexplosion.com/Febs/Rockbox/apple_dynamic_range.png)
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: Febs on August 26, 2006, 09:55:49 PM
I have seen speculation that the decoders in the respective firmwares could cause differences in sound, so I encoded the RMAA test sound file from .wav to .mp3 using LAME 3.97b2 -V2.  All other test conditions are the same as above.

Rockbox frequency response (MP3, headphone output):


(http://jazzexplosion.com/Febs/Rockbox/rockbox_mp3freqres.png)


Rockbox frequency response (MP3, line output):


(http://jazzexplosion.com/Febs/Rockbox/RB_ipod_lineout_mp3.png)

Apple FW frequency response (MP3,headphone output):

(http://jazzexplosion.com/Febs/Rockbox/apple-mp3freqres.png)

The RMAA test uses a sine wave sweep, which is notoriously difficult for mp3 encoders to handle, so I'm not sure how valuable this test is.  Nevertheless, it is interesting to see that the results are substantially similar for both Rockbox and the original Apple firmware.
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: lachlan on August 26, 2006, 10:12:48 PM
I would go as far as to say they are almost identical. Out of curiosity, would it better to have the line showing greater changes, or being smoother?
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: omendelovitz on August 27, 2006, 01:50:30 AM
Hey Febs, a quick Q 4 U.  What other measures of sound output would one consider in assessing SQ?  I ask b/c my main beef w/ the original OS is that it seemed to sound more "congested" than Rockbox, i.e. worse instrument separation, resolution (layering) and imaging.  I'm also wondering about the background hiss audible in AOS w/ high sensitivity IEM's.  

I'm not trying to be facetious here, just want to learn...
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: aegis on August 27, 2006, 06:06:55 AM
Hi Febs,

I'm no engineer but these graphs look pretty the same and the tiny differences can be just statistically irrelevant so I would ask: how much are frequency response and dynamics just a question of hardware - quality of filters, circuits etc?

If the mp3 file is the same the only difference between systems is the decoder used by particular players and equallizer presets. Can someone explain if it has anything to do with frequency response assuming codec is not just broken?
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: Febs on August 27, 2006, 10:27:29 AM
I think that these are all good questions, but I am no engineer either.  I would love to get input from some of the more technical people here as to how these results should be interpreted.

omendelovitz, are you certain that you are comparing the two with the exact same volume and with no processing or EQ involved?

I've added a frequency response graph of Rockbox/iPod with the line-out to the second post above.
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: omendelovitz on August 27, 2006, 10:35:05 AM
Yeah, Febs I am.  I should Qualify last night's statement though.  Truthfully, using a VERY transparent Amp and High End Cans, I played my reference tunes through both RB and AOS, one after another (admittedly, with a pause = reboot time).  My new conclusions are:  RB makes iPod sound more like ZVM - wider soundstage (and almost by extension, better imaging - to much to explain and I'd probably mess it up, so I won't go there); otherwise, the two sound the same.  A proper analog for this scenario would be Grado sound vs. Sennheiser sound.  

Hope that helps.
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: Febs on August 27, 2006, 10:47:35 AM
Unfortunately, I could see this thread getting bogged down with debate regarding wholly subjective impressions of sound quality, and since there are other existing threads on that, I think that we should not go there.  The whole purpose of this thread was to take an objective look at the differences between the firmware.  So, for purposes of this thread, let's stick with objective analysis.

(I will say this:  if you're going to analogize between the Sennheiser and Grado sound, I think that even the staunchest subjectivist would agree that the Sennheiser "veil" should be observable in the frequency response.)
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: breez on August 27, 2006, 10:57:33 AM
Have you tried the stereo crosstalk test in RMAA and found any differences between firmwares?
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: omendelovitz on August 27, 2006, 10:58:57 AM
I agree, but I don't think Sennheiser has a veil.  In that case, let's suffice to say that RB sounds more like a concert hall or stadium in terms of soundstage width, and AOS sounds more like being on stage at a rock concert or small club.  Like I said earlier, the difference in sound is comparable to ABX'ing a creative Zen Vision M (line out) to the AOS (line out).  i.e. RB presents the soundstage the same as Creative does in the Zen Vision M.

'nuff said ;)
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: Febs on August 27, 2006, 11:05:46 AM
Quote from: omendevolitz
Like I said earlier, the difference in sound is comparable to ABX'ing a creative Zen Vision M (line out) to the AOS (line out).
omen, I would agree with you if your comparison had involved an ABX test.  I think that it would be *very* interesting to do an ABX test.

Quote from: breez
Have you tried the stereo crosstalk test in RMAA and found any differences between firmwares?
I had RMMA run all of the available tests.  Give me a minute and I will post the saved RMAA files here.

Edit:  Here they are:

iriver headphone out, Rockbox (http://jazzexplosion.com/Febs/Rockbox/Rockbox%20(iriver%20headphone%20out)%2016-bit,%2044%20kHz.sav)
ipod, headphone out, Apple fw, mp3 (http://jazzexplosion.com/Febs/Rockbox/iPod%20-V2%20MP3%2016-bit,%2044%20kHz.sav)
iPod headphone out, Apple fw (http://jazzexplosion.com/Febs/Rockbox/iPod%2016-bit,%2044%20kHz.sav)
iPod headphone out, Rockbox, mp3 (http://jazzexplosion.com/Febs/Rockbox/iPod%20Rockbox%20-V2%20MP3%2016-bit,%2044%20kHz.sav)
iPod headphone out, Rockbox (http://jazzexplosion.com/Febs/Rockbox/iPod%20Rockbox%2016-bit,%2044%20kHz.sav)
iPod line out, Rockbox (http://jazzexplosion.com/Febs/Rockbox/iPod%20Rockbox%20line%20out%2016-bit,%2044%20kHz.sav)

You should be able to open these files in RMAA and view all of the results.
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: saratoga on August 27, 2006, 12:46:57 PM
Yeah, Febs I am.  I should Qualify last night's statement though.  Truthfully, using a VERY transparent Amp and High End Cans, I played my reference tunes through both RB and AOS, one after another (admittedly, with a pause = reboot time).  My new conclusions are:  RB makes iPod sound more like ZVM - wider soundstage (and almost by extension, better imaging - to much to explain and I'd probably mess it up, so I won't go there); otherwise, the two sound the same.  A proper analog for this scenario would be Grado sound vs. Sennheiser sound.  

Hope that helps.

Try an ABX test.  The stereo seperation between the two is virtually identical, so you shouldn't be hearing anything at all.  My guess is the delay between listening to the two samples and the placebo is effecting you.

Quote from: aegis
I'm no engineer but these graphs look pretty the same and the tiny differences can be just statistically irrelevant so I would ask: how much are frequency response and dynamics just a question of hardware - quality of filters, circuits etc?

Its basically all hardware, at least for lossless tests.  You'll only see a difference if something is very wrong (like we did on the X5 a while back when it was accidently put in 22.05k sample rate instead of 44.1k).  Thats why the Rockbox and Apple firmwares perform the same for the WAV test.

The lossy tests are effected by the decoder.

Quote from: omendelovitz
What other measures of sound output would one consider in assessing SQ?

Noise floor will tell you how bad the hiss on the system will be.  Frequency response will tell you how much EQ you'll need.  THD and IMD will tell you how much distortion there in the system.  Crosstalk tells you how well stereo works.  No one of them alone is more important then the rest, it just depends on what you need to know.

Also, be careful with the frequency response plot.  That is valid ONLY for something with the same load impedance as the test setup.  For a sound card, the impedance will be VERY high, while typical headphones are very low.  So if you want to test frequency response as you'll hear it, you should test with your specific headphones in parallel with the sound card (doing this on the ipod gives surprisingly different results with <32 ohm headphones too).
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: omendelovitz on August 27, 2006, 02:00:02 PM
If anyone is in the Toronto, Canada area and has a 5G ipod they want to loan me and/or come test with me, I'd be glad to ABX.  As it stands right now, I don't have that kind of equipment.  I'd love to try this option as well.
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: saratoga on August 27, 2006, 02:20:58 PM
If anyone is in the Toronto, Canada area and has a 5G ipod they want to loan me and/or come test with me, I'd be glad to ABX.  As it stands right now, I don't have that kind of equipment.  I'd love to try this option as well.

If you don't have an Ipod, how did you manage to do the previous test?

Edit:  Do you mean you want two ipods to do an ABX test?  Thats not needed, or even a good idea.  Just record the output of the firmwares using your sound card.  Then load them up in any wave editor and make sure to clip the wavs so they start and end on exactly the same point in the song.  FInally, load into any ABX program (foobar w/ ABX plugin, winabx, HR/ABC, etc).  Since the output appears to be identical, you should get essentially identical recordings.
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: omendelovitz on August 27, 2006, 02:28:39 PM
Got it.  Actually, I was thinking of two ipods set to the same settings and using the same files/equipment.  honestly, I don't think RMAA tells the entire audio picture.  With all that we've been talking about in this thread alone, we haven't mentioned how we'd measure "soundstage" presentation, never mind imaging and/or resolution measurements.  AFAIK, those audio cues can't be measured (if so, someone please correct me and teach me ;D)
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: Febs on August 27, 2006, 02:58:39 PM
What exactly is "resolution"?
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: omendelovitz on August 27, 2006, 03:00:52 PM
What exactly is "resolution"?


resolution means instrument separation
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: saratoga on August 27, 2006, 03:02:17 PM
With all that we've been talking about in this thread alone, we haven't mentioned how we'd measure "soundstage" presentation,

Sound stage == stereo crosstalk.

never mind imaging

Within this context, imaging and sound stage and stereo crosstalk are the same thing.

and/or resolution measurements.  

Resolution for all of these is 16 bits.  You don't actually need to measure that.

AFAIK, those audio cues can't be measured

All things in nature can be measured.
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: Febs on August 27, 2006, 03:02:54 PM
Quote from: omendevolitz
resolution means instrument separation
Then what is "imaging" and how does it differ?
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: Febs on August 27, 2006, 03:06:12 PM
Actually, never mind.  These are futile questions and exactly the type of debate that I did not want to get into.

Omendevolitz, what do you use as your "reference" files?  

Do you use Foobar?
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: omendelovitz on August 27, 2006, 03:15:48 PM
imaging is the dynamics of spatial location - the 3-D'ness of audio; depending on your system, you may be able to 'follow' the music around in your head and pinpoint an instrument's virtual location, or a progression b/w channels as a particular sound moves 'through' its audio field.  

Using a headphone example - assume imaginary horizontal lines are extending in 3 directions from the centre of your head.  Label them x,y,z arbitrarily.  A well recorded (engineered) musical piece will 'locate' a sound in a  certain location (e.g. - left 3 on the x, -5 on the y and -3 on the z axes) rather than placing it all right up 'against' your ear.  Think of watching real life vs. television or imax v/s the movies.  TV and regular movies are flatter.  Hope this helps.
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: omendelovitz on August 27, 2006, 03:18:45 PM
my reference files are songs that I know well that have been recorded well enough to notice 'placement' of insturments, etc.  Some are in ALAC, others in AAC 320 or Lame 320.

I understand you not wanting to delve too deeply into this on this forum/thread.  These 'conversations' are better left to head-fi, etc.
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: Febs on August 27, 2006, 03:26:43 PM
What are the actual songs that you use as reference songs?
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: omendelovitz on August 27, 2006, 03:52:28 PM
Here you go:

The Be Good Tanyas - The Littlest Birds

Django Reinhardt - Minor Swing

Chris de Burgh - Spanish Train

Dave Matthews Band - The Dreaming Tree

Dave Matthews Band - Listener Supported

Dire Straits - Money For Nothing

Dixie Chicks - Landslide

Earth, Wind & Fire - September

Genesis - Jesus He Knows Me

George Michael - Fastlove

Goo Goo Dolls - Slide

Various Artists, Santaolallia Gustavo - De Usuahia A La Quiaca

Heart - Crazy On You

Heart - 15-Back to Avalon

James Taylor - Carolina In My Mind

The La's - There She Goes

Paul Simon - You Can Call Me Al

Paul Simon - Crazy Love. Vol 2

Phil Collins - Easy Lover

Phil Collins - Something Happened on the Way to Heaven

Red Hot Chili Peppers - Higher Ground

Rush - 21 - 2 Overture - The Temples Of Syrinx

Rush - The Spirit Of Radio

Sloan - Delivering Maybes

Sloan - The Other Man

The Smiths - Bigmouth Strikes Again

The Smiths - This Charming Man

STYX - Mr. Roboto

The Tea Party - The Badger

The Tea Party - Shadows On The Mountainside

Toad the Wet Sprocket - Whatever I Fear

Toad the Wet Sprocket - Nanci

Van Halen - Hot For Teacher

Dave Matthews - So Damn Lucky

Dave Matthews - Trouble

Dave Matthews Band - Stay (Wasting Time)

Pat Metheny - Sisters

Pat Metheny - New Chautauqua

Pat Metheny Group - James

Pat Metheny - Above the Treetops

Pat Metheny - Facing West

Pat Metheny - See the World

Pat Metheny Group - Part 1

Dave Matthews Band - Say Goodbye

Dave Matthews Band - Tripping Billies

Dave Matthews Band - The Stone

Phil Collins - I Cannot Believe It's True

Dave Brubeck - Take Five

Dead Can Dance - American Dreaming
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: omendelovitz on August 27, 2006, 04:00:45 PM
Saratoga,

While I appreciate your enthusiasm, Platitudes don't help.  I'm hoping you can be a little more specific with your statements - e.g. how does stereo crosstalk effect perceived spaciousness of the soundstage?  How does it affect imaging?  exacly what is 16 bit?  How would it affect instrument separation?

oh, and let me correct myself - I've never found an audio hobbyist whose been able to measure the above types of audio cues; I've also never had a chance to speak/interact with someone who works with these issues professionally to teach me such nuances in audio reproductions - if you have, please fill me in or at least refer me to places where I can learn all these types of goodies for myself! Yumm
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: Febs on August 27, 2006, 05:25:01 PM
Here you go:
...

Dixie Chicks - Landslide

Omendelovitz, since you hear a difference between the Apple FW and Rockbox, I would be greatly appreciative if you could do a test for me.  I have posted two clips of Landslide (one of your reference tracks) for you to compare:

Wave files:
Clip 1 (http://jazzexplosion.com/Febs/Rockbox/Landslide1.wav)
Clip 2 (http://jazzexplosion.com/Febs/Rockbox/Landslide2.wav)

WavPack Files

Clip 1 (http://jazzexplosion.com/Febs/Rockbox/Landslide1.wv)
Clip 2 (http://jazzexplosion.com/Febs/Rockbox/Landslide2.wv)

It doesn't matter which set you choose; they are identical except that obviously the WavPack files are smaller.  To prepare the clips, I ripped Landslide from CD using EAC.  The .wave file was placed on my iPod, and then played back using both the Apple firmware and Rockbox.  In each case, the clips were recorded to  .wav file, using the iPod's line out and a pocket dock.  In each firmware, the iPod's volume was set on maximum (i.e., +6dB in Rockbox; a "full blue line" in the Apple FW).   No EQ or other processing was enabled.  I did no post-processing other than to (1) peak normalize both files so that there would be no volume differences to skew the results, and (2) trim the starting and ending points of each clip so that as near as humanly possible, they were exactly the same length.

Please compare the two clips using an ABX program such as the one built into Foobar2000.  You should do 12 trials.  Turn off the option allowing you to view the results of each trial as the test is in progress.  When you have finished 12 trials, exit the ABX program (being sure to save the results) and post the results here.

Thanks!  Please let me know if you have any questions.
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: saratoga on August 27, 2006, 07:42:10 PM
Saratoga,

While I appreciate your enthusiasm, Platitudes don't help.  

Platitudes are statements such as "life is great" that don't actually mean anything.  I posted definitions of those terms, as commonly used.

I'm hoping you can be a little more specific with your statements - e.g. how does stereo crosstalk effect perceived spaciousness of the soundstage?  How does it affect imaging?

Stereo cross talk collapses the sound stage.  At 0 dB you have mono.  At negative infinity you have perfect channel speration.  In this case though, since the noise floor is roughly the same as the cross talk, its a safe assumption that the channels are not audiably coupled.

Regarding imaging verses sound stage, they really are the same thing in this context.  Heres the first random site I found googling:

http://www.electronixwarehouse.com/education/glossary/S.htm

"
Soundstage
...
 (See also Imaging above)
"

So it seems its not just me who uses them interchangably.

 exacly what is 16 bit?  How would it affect instrument separation?

The resolution of the PCM audio is 16 bits in Rockbox and Apple firmwares.  It doesn't effect imaging, it determines how well the system can resolve amplitude changes in reconstructed waveforms.  This in turn determines the dynamic range of the system.  16 bit systems typically have a maxium of 96 dB SNR.


oh, and let me correct myself - I've never found an audio hobbyist whose been able to measure the above types of audio cues; I've also never had a chance to speak/interact with someone who works with these issues professionally to teach me such nuances in audio reproductions - if you have, please fill me in or at least refer me to places where I can learn all these types of goodies for myself! Yumm

A lot of audio terminology is simply made up by hobbyists as the go along.  Its very common on the internet for people with no real understanding of audio to hide behind the equipment they use and the nebulous terms they throw around (I'm not singleing you out).  Its best to just avoid fluffy language like that entirely and say what you mean.  At least thats my preference.  

But yeah, take the ABX test and post your results.  Either way it'll be interesting.
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: saratoga on August 27, 2006, 07:49:40 PM
Hey Febs, I think you mean

http://jazzexplosion.com/Febs/Rockbox/Landslide1.wv
http://jazzexplosion.com/Febs/Rockbox/Landslide2.wv

are the wavepack files.

Anyway, thats a difficult test.  I tried but could not ABX it.  Though it could be that I'm not sure what to look for, which doesn't help.
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: Febs on August 27, 2006, 09:02:14 PM
Thanks, I fixed the links.
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: travishayes89 on August 27, 2006, 09:06:01 PM
i have not run such tests, but i have listened to music through rb on my ipod (before it got dunked) and also on the apple os, and noticed that the rb output sounded better on certain songs
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: ryran on August 27, 2006, 09:37:18 PM
No way in hell I could abx those two (I gave it a try).

[In other news: in la futura, why not just use wavgain to get the samples to exactly the same volume?]
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: omendelovitz on August 27, 2006, 10:13:25 PM
Febs, Saratoga,

EDIT:  After checking all my sound settings, I realized that while I thought I had all sound effects off, it turns out that in fact I didn't.  Upon a preliminary re-listen, I am finding a diminishing difference b/w the two OS'.  OOPS!  How embarrassing...  Anyway, I will listen to your ABX samples, as soon as I learn how to ABX ???  

Saratoga, thanks for the details; Febs, thanks for the tests - anywhere you know where I can find instructions on Foobar ABX'ing?  I'll check the help file first I guess.
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: soap on August 28, 2006, 05:57:35 AM
(Assuming Foobar 0.8.3) (Assuming you have the ABX component installed) (Believe it comes with the Special and Full builds)
1- Drop the two files into Foobar.
2- Select the two tracks, right click, and from the context menu select "ABX two tracks"
3- Foobar will do its thing preparing the files, then will present you with the ABX comparator screen.  From here you can listen to the "A" sample, the "B" sample, the "X" sample, and the "Y" sample.  The letters assigned to your two tracks will be randomly changed each trial, your task is to play the files however you want and tell if A=X or if A=Y.
The prefered methodology is to check the "Hide results" selection box, and to do at least 12 "trials" before exiting and saving your results as a text file.
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: XavierGr on August 28, 2006, 06:47:37 AM
Bravo Febs! Magnificent work on the samples and analyzing both firmware output!
This is a definite benchmark to shut off the mouths of some members here and MisticRiver.

As many of you have noticed, Rockbox is being under "attack" as having "inferior" sound quality compared to original firmware. (see the first golden quote in wiki: http://www.rockbox.org/twiki/bin/view/Main/GoldenQuotes)

For once again these persons are proven wrong. These tests show that Rockbox differences from original firmware are inaudible.

Again, thank you Febs, for providing the most failsafe way to compare sound quality and for the analysis that you made.
Let’s hope that this debate will end, and Rockbox will no longer be under the "inferior-sound-quality" myth.

PS: I will wait for omendelovitz's ABX test before continuing my "triumphology", who knows....  ::)
Title: And the winner is...
Post by: omendelovitz on August 28, 2006, 07:52:37 PM
Science overall.  I have embraced my inner scientist and resigned from audiophiledom, possibly permanently.  

Just a few quips here before I move on to my results:

1 - Foobar's ABX is an absolute wonder!  I love this program.  It is the cream of software, forget the fact that it is freeware...

2 - There is truly nothing better than being able to test one's own biases and perceptions in this manner;

3 - I can't wait for Rockbox to come out in full release for the iPod.  As a FW overall, it really trumps Apple.

Now for my results [drum roll please] - with my soundcard, high end headphones and crystal clear and very transparent amp I...

can tell no difference b/w the two wav files febs so graciously posted online for me.  Now, before you start triumphonolizing, Xavier, I must add a "scientist's" warning to my findings - I would personally claim that therefore there is no audible difference b/w the two firmwares, using Febs' readings in conjunction with my observations, but in reality, my soundcard is internal and therefore isn't all that great.  I will also add that when comparing lossless to mp3 alt insane, I could hear no difference on the same setup.  Take that for what you will.  

What I will say [finally] are two last things - 1) kudos to Febs for his work and perseverance; 2) if anyone wants to make any claims re: RB SQ to AFW, just point them to this thread and tell them to put out or shut up! And then ignore those people, b/c they're going to see what they want to see (and disregard the rest... thx for the quote, Paul Simon!).

Over and Out - Omendelovitz
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: Atheistic Freedom on August 29, 2006, 12:39:40 AM
Yeah, but turn on Hardware EQ and Software EQ Precut to -12 dB...  ;)

Worlds better.
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: XavierGr on August 29, 2006, 09:29:14 AM
Omendelovitz, thanks for taking the test. You are the first one to take it, that is admirable.

Of course I didn't mean any direct offence at you. You said that finally, it was a weird EQ setup that made you believe Rockbox sounded different.

Anyway, I am a happy man now!

P.S: I am so happy that I will sticky this.
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: Llorean on August 29, 2006, 10:23:19 AM
Stickying should be reserved for forum business. If sound quality comparison needs to be put in a permanent place, it should be made into a wiki page.

A link to that wiki page could be incorporated into a playback FAQ, but this isn't really something that belongs stickied on its own, as it'd be best if there were a minimum of stickies.
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: wolfB on August 29, 2006, 12:27:26 PM
I'd agree the soundquality of the dock output from a 5G iPod can be considered equal. But I can definitely recognize a difference from the headphone out, at least with my sensetive IEMs (UE-10Pro and super.fi 5Pro). Quite possibly an impedance issue, as it might be related to the different hissing noise floor, which is significantly lower in Rockbox.

So I'd like to know two things:

1. Why is there apparently much less hiss noise floor on the 5G Ipod's headphone out with low impedance IEMs?

2. Would it be possible to ABX  two files, recorded from the iPod's headphone jack?

Maybe Febs could provide those two files???
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: saratoga on August 29, 2006, 09:41:04 PM
I'd agree the soundquality of the dock output from a 5G iPod can be considered equal. But I can definitely recognize a difference from the headphone out, at least with my sensetive IEMs (UE-10Pro and super.fi 5Pro). Quite possibly an impedance issue, as it might be related to the different hissing noise floor, which is significantly lower in Rockbox.


Do you mean the difference between the headphone and line out jacks?  That would be expected, the line out has no amplification and a higher impedance, so it should have a slightly lower noise floor.

If you mean a difference between firmwares, that shouldn't happen.

Quote
2. Would it be possible to ABX  two files, recorded from the iPod's headphone jack?

Yes.  Hook up the Ipod to your computer and see for yourself.  However, if you are concerned about impedance, then you MUST also hook up your headphones in parallel with the mic jack in order to provide a load for the headphone jack on the Ipod (otherwise you'll just test the unloaded noise floor, which you suggested is not the same).
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: wolfB on August 30, 2006, 02:11:14 AM
If you mean a difference between firmwares, that shouldn't happen.

Yes, that is exactly what I meant. To be clear, the scenario is as follows:

Low impedance high sensetive IEM on the headphone out of the 5G iPod.

Apple-FW: clearly audible hissing noise floor.
Rockbox: definitely much less hiss.

I am not sure if this is due to impedance miss match, as the hardware didn't change.

That's what is objectively reproducable. On a subjective side note I'd say the overall soundquality is improved on headphone output with Rbx (I know, I am not alone with this...see head-fi forum), IMO at least on the same level as from the dock out.
To shed some light on this subjective impression it would be great, if this could be checked in a ABX test with two files recorded from the headphone output.

I doubt I have the appropriate hardware to record the files, but I'll try.

Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: saratoga on August 30, 2006, 08:07:08 PM
If you mean a difference between firmwares, that shouldn't happen.

Yes, that is exactly what I meant. To be clear, the scenario is as follows:

Low impedance high sensetive IEM on the headphone out of the 5G iPod.

Apple-FW: clearly audible hissing noise floor.
Rockbox: definitely much less hiss.

I am not sure if this is due to impedance miss match, as the hardware didn't change.


That definately sounds fishy.  Could you confirm it with either ABX or RMAA results?  

I doubt I have the appropriate hardware to record the files, but I'll tr

You only need a PC + audio cable, so hopefully thats not too big a problem.

Edit:  Well I suppose if you think the issue is the load induced by your headphones, then you need a headphone splitter so that you can have both the headphones and PC listening at once (or an audio cable that you were willing to cut and splice into).
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: wolfB on August 31, 2006, 04:11:31 AM
Thanks for the tips, saratoga.

I'd suppose that it could be confirmed with ABX.

I'll try to record two wav files, and will post detailed results here. Just give me two or three days.

Edit: I am obviously unable to record decent wav-files, as I get rather destorted files with overemphased bass only. iPod Headphone out -> LineIn of nVidia onboard soundcard, WindowsXP, Audio Recorder 3.0. All EQ-settings flat or disabled, volume on the iPod with Rbx tried at -25db, 0db, and +6 db, but no avail. I'd think a software recorder with a peek-meter would be helpfull. Could anybody point me to a freeware download, please.

 
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: Febs on September 01, 2006, 11:59:04 AM
Goldwave or Audacity would do it.

Also:

1.  Check your soundcard to make sure you are plugging into the line-in, and not the microphone in.

2.  If your sound card does not have separate inputs, check your soundcard's set up program to see if there is a way to set it to line-in in software.

3.  If not, use the volume mixer to turn down the input volume on the mic/line input.
Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: wolfB on September 01, 2006, 01:27:47 PM
Thanks Febs.

I found the shareware software LP Recorder, which worked quite well. Unfortunately I could not avoid a static background noise from my harddrive, audible in both files. Also, the above mentioned hiss of the Apple-FW unfortunately seems not to be audible on these files, though I used a splitter and hooked my super.fi's on during the recording. All other settings where as flat as they could be, No EQ, No Gain, No Nothing...so, let's hear, if you can spot an audio-relevant difference between these recordings.

Here are the two files, compressed to FLAC, and each is about 5MB.

record_1.flac (http://www.muenster.de/~w.beinha/temp/DOWNLOAD/record_1.flac)
record_2.flac (http://www.muenster.de/~w.beinha/temp/DOWNLOAD/record_2.flac)

Title: Re: Rockbox Audio Quality versus Original Firmware (RMAA comparison)
Post by: fabolousrmx on September 11, 2006, 02:15:31 PM
I am very interested in this wolfB, i am in the same situation that you are in. Using my er6is  in the appleOS there is a hiss, but in rockbox i cannot detect any his what so ever after everything is loaded.