No, it doesn't work correctly on all players. What probably caused the confusion here is that it is only halfway broken or halfway working - depends on your point of view.

"Follow playlist" works as long as the music is playing or paused - but it does not work when playback is fully stopped (as the bug report pondlife mentioned states). When stopped you'll be dropped back to the last selected track or when you haven't selected anything since the last reboot and just resumed then you'll end up in the root directory.
I hope this bug will be fixed one day (though it already exists for a while) but currently there is no way around this, that I know of.