Foobar2000 does support .m3u8, it's just a bit picky about the format. The file must contain a BOM and the path separator char must be "\"...
No, it is not necessary for the separator to be "\". It was my mistake to still use an exported playlist from mediamonkey(m3u) for testing which i renamed to "m3u8" and added a track with the character ñ afterwards.
the result was that this playlist was still saved with the coding "8 Bit" instead of "UTF-8" and "ñ" instead of "ñ" in the title of the added song which i noticed when i opened the playlist in the text editor and and took a look in the coding options.
After changing the coding to "UTF-8" and saving the playlist in the editor the "ñ" became "ñ" again and i can open it in the foobar2000 without problems.
And even more interesting: renaming the playlist to "m3u" does not seem to make any difference for rockbox. When the playlist is a UTF-8 one i can add the tracks with special characters as well and they do not get change, and there is no problem to import or open it in mediamonkey either :-)
Thanks a lot once more for your help!