I'm certainly no hardware expert, but I would hazard a guess that the older players used dedicated hardware decoders because the processors available weren't powerful enough at the time to do software mp3 decoding. The limitations of a dedicated hardware decoder is that it *only* supports the mp3 format (though I think .wav files can be played back as well, as they're already PCM), and some times the bitrates supported are fixed.
Software decoding allows for more supported formats (some common ones being OGG Vorbis and WMA), and Rockbox has written and ported codecs for less commonly supported formats (such as FLAC and SPC, for a full listing go
here.) I would expect that sound quality between a dedicated hardware decoder and a software codec would be fairly equal, as the quality of the mp3 would really depend on its bitrate.