Correct.
Actually, it first adds a header line (I'm doing this from memory, but I'm 99% sure it's correct) that says:
#EXTM3U
Then, before each file, you have the following
#EXTINF:Length,Free Text to display
where Length is in seconds and then a nice name.
Note that Rockbox seems to successfully ignore these lines, which is good.
I was hoping that it would know to use them for a nice display in the playlist (I use an iPod with tagcache, and so my file names are things like F03/TRMI.mp3).
I looked into added support myself... it doesn't seem to bad, but when I arrived at work today, I found that I have a bunch more work than I realized!