Large oggs with chained streams

[nimdae]

I have a large amount of 40 to 50 mb ogg files that consist of anywhere from 4 to 8 chained streams. When I attempt to play them, nothing happens. My music ogg files play just fine, however. Is there a limit to the size that an audio file will play? I can unchain them to have smaller ogg files, but due to the number of files and the reason for chaining them, I don't really want to do this.

[Llorean]

Out of curiosity, what's the reason for chaining them?

Chained oggs are not supported by Rockbox at this time.

[nimdae]

They are radio talk shows with the commercials cut out. Each logical stream is a segment between commercials. I chain them so I can randomize the shows and not the segments.

I could probably re-encode them, but there's hundreds of them. They are used for a stream that I share with my friends, but I wanted to put them on my ipod so I could travel with them.

Knowing quite a bit about the ogg format, I may just screw with the headers and ogg pages, creating a single physical stream.

[saratoga]

You could also add support for chaied ogg files in Rockbox!  

[nimdae]

Heh, I'm not much of a programmer, to be honest

One thing to note is most linux ogg players don't play chained oggs either. They will play the first logical stream, then when that stream ends, looks for the next page for that particular logical stream, which won't exist. Rockbox won't even play the first logical stream.

[nimdae]

While I'm not much of a C programmer (or -insert language used in rockbox-), I do know a bit of the ogg page header format as I mentioned earlier. If someone wants to take this information and use it in the ogg decoder in rockbox, it would allow the addition of using chained streams.

Oggs consist of physical and logical streams. Most people will only ever deal with one physical and one logical stream in a file or other stream. This makes writing a player easy because when the file changes, all counters reset.

Each logical stream has a serial number. Every page in the stream contains that serial number to identify it as part of that stream. The ogg decoder reads the serial number (or should anyway) and the page number to make sure we playing the right audio for the right stream. If the serial number changes, at least in a chained stream, the page sequence counter should reset, as if it is a new file.

What made me learn about the ogg page header format is a while back I used the jetaudio ogg encoder to encode my radio talk show streams. I discovered my streaming software was having errors with the ogg files. I discovered that jetaudio was setting the serial number on all files to 0x00000000. I wrote a script to fix all of the files with random serial numbers, adjusting the necesary fields in the header as needed.

I'm now modifying the script to see if I can include a page sequence change as well and see if it works. I'm thinking it will likely work, with the logical stream headers just being read as "bad" data, stream update data, or something...at least I HOPE it will work in, say, winamp, then I'll test in rockbox

[nimdae]

I just finished my ogg patching script, it doesn't make a perfect ogg, but it plays. I guess I'll be patching the rest overnight if I find the playback on rockbox satisfactory.

[lowlight]

Have you tried oggsplit?
http://www.rarewares.org/ogg.html

[nimdae]

To be honest, I could unchain them easily. However, that's something I want to avoid as it would defeat what I was trying to accomplish by chaining them. The fix I made works, so I'll use that globally. I'm leaving the original files alone so that I have undamaged oggs. The ones going on rockbox are technically damaged, but they play, and that's what matters to me. It only takes about 2 and a half minutes to "fix" a file so this will be done before I wake up in the morning This is just for me so no big deal if it works for me.

[Lear]

While I'm not much of a C programmer (or -insert language used in rockbox-), I do know a bit of the ogg page header format as I mentioned earlier. If someone wants to take this information and use it in the ogg decoder in rockbox, it would allow the addition of using chained streams.

Decoding the page headers is the simple bit. Various things about how Rockbox works make it harder to fix. The main issues are these:

• Tremor (the decoder) wants to seek through the whole file in order to enable seeking. Not good when buffering part of a file, as it would force a full rebuffer when starting to play such a file. We hack around this, but this hack is limited to one stream only, because of the next reason...
• Because there is no (real) dynamic memory allocation in Rockbox, we can't just allocate room for n streams and pass that information to the Vorbis decoder (and use the hack to avoid Tremor's seeking). If the metadata was put in the file buffer, it should be possible to avoid that limitation.
• Rockbox do support a limited form of memory allocations in the codecs, but it isn't good enough for what Tremor needs for chained streams. It needs a partial free in this case, and that isn't supported. (This could possibly be hacked around.)
• Regardless of the above, the rest of Rockbox just doesn't support the case of multiple tracks in one file (with metadata, seeking, track skipping and stuff like that). (This is only UI annoyances, and wouldn't prevent playback.)

[pabouk]

26 Nov 20:56   Tomasz Malesinski
apps/codecs/Tremor/SOURCES 1.3
apps/codecs/Tremor/oggmalloc.c 1.1
apps/codecs/Tremor/os_types.h 1.5
apps/codecs/Tremor/sharedbook.c 1.7
apps/codecs/vorbis.c 1.27
FS#6357, patch 3: implemented simple temporary malloc for the Vorbis decoder.

Is this malloc implementation sufficient to enable playback of ogg files with chained streams? Could not the initial implementation circumvent Lear's last point by treating the multiple streams as a single track?

Some internet radios use ogg with chained streams to be able to change the song title during playback. Unfortunately such captured broadcasts cannot be played in Rockbox.

[Lear]

Is this malloc implementation sufficient to enable playback of ogg files with chained streams? Could not the initial implementation circumvent Lear's last point by treating the multiple streams as a single track?

No, that malloc is for temporary things within Tremor, typically where a function allocates some memory, do a bit of processing and then free the memory again. One of the problems here is that Tremor will allocate memory for parsing/buffering the file ("Ogg buffers") as well as various stream data ("Vorbis buffers"). When changing streams, all Vorbis buffers need to be freed, but the Ogg buffers should be kept.