AVI playback

[sirdan]

Hello, sorry for reviving old thread but I thought recently about playing mpeg-4 videos in rockox and that's what has come to my mind:

Actually i find some advantages for it over mpeg2. Many mobile phones have feature to record videos in .mp4 container that actually contains mpeg-4 Single Profile video format and amr/aac for audio. So Rockbox could play that low resolution recordings out of the box!! . So no transcode here. Using mpeg-4 Advanced Single Profile (xvid/divx) vastly decrease amount of needed space comparing to mpeg2, more visible when compressing full videos.
Rockbox actually recognizes mp4 container (but mobiles produce some kind of hybrid 3gp/mp4). And theres is also need to port decoder from xvid or from ffmpeg project.

Advantages:
- many mobiles recordings actually could be played without transcoding,
- many compression programs out there,
- high compression ratio - we save at minimum half the space needed for mpeg2 clips
   Im able to squizze 5 min clip into 5 MB file or so, incuding 32bit/s mono aac. Thats 176x144 24fps around 110kbit/s for my SE k600i with acceptable quality.
   Elephant Dream mpeg2: 11min 224x176 24fps 450kbit/s = 35 MB file
   Elephant Dream xvid no b frames : 11 min 224x176 24fps 185kbit/s = 15 MB     >>Quality is worse and image is less sharper just a little as i reencoded previous mpeg2 ED (in megui HQ 2-pass mode). If RB would decode b frames, quality of encodes increase and cpu usage as well.

Disadvantages:
- higher cpu usage
   My k600i has ARM9 ~103MHZ processor and plays 176x144 mp4's.
   Logically, we can decrease cpu requirements, lowering fps for encoded video. Going from 24fps to 12fps can make   decoding faster almost twice(?) and on my k600 video is still smooth with that frame rate. There is no big smoothing impact on so small lcd's when lowering frame rate i think.
   Also compressing video without b frames reduce cpu decoding requirements.
- need someone to code it

What do you guys think about this?

[AlexP]

What do you guys think about this?

The same as always - if someone were to code it (including all the associated stuff like making mpegplayer multi-codec capable) then I'm sure it would be appreciated and go in

[saratoga]

   Elephant Dream mpeg2: 11min 224x176 24fps 450kbit/s = 35 MB file
   Elephant Dream xvid no b frames : 11 min 224x176 24fps 185kbit/s = 15 MB     >>Quality is worse and image is less sharper just a little as i reencoded previous mpeg2 ED (in megui HQ 2-pass mode). If RB would decode b frames, quality of encodes increase and cpu usage as well.

As you can see if you disable MPEG4 features its not much better then MPEG2.  Unfortunately, theres no such thing as free lunch and enabling better MPEG4 compression features means more CPU usage which means lower frame rates.

As our targets get faster MPEG4 codecs start to make more sense, but its a lot of work to benefit a small number of people.  If you're interested, we'd love to see more codecs in Rockbox, just don't expect everyone else to do the work

[sirdan]

As you can see if you disable MPEG4 features its not much better then MPEG2. 

Well too much said imho I talk about disabling only b frames. With I and P frames, encodes are twice smaller at minimum. With b frames even more. And i think we still can go lower.

As our targets get faster MPEG4 codecs start to make more sense, but its a lot of work to benefit a small number of people.

Yes, main problem is that we cannot estimate of how decoding of mpeg4 will be cpu demanding. I really dont want that, when someone spent much time on coding and it occured that videos cant play in realtime.
The edge is 80 MHZ dualcore (and up) that sit in more than a half of targets. But i see room for that especially for 16:9 format. Applying some tweaks to encoded videos may allow to realtime decode:
- no b frames
- lower fps for encodings (i hope the smoothness wont be sacrified)
- 16:9 format
- some decoder optimizations
- last resort is to lowering resolution.

Sorry if that all sounds silly but the truth is that mpeg-4 Single Profile (or h.263 in .3gp) is supposed to be the format for portable devices not mpeg-2. Thus i assume the former shouldnt be such decoding demanding as we think .

If you're interested, we'd love to see more codecs in Rockbox, just don't expect everyone else to do the work

I know guys, i know... if i would just can coding

[dar02]

For example Ipod 5G Video can play unscaled mpeg4 in .mp4 container but with it's orginal Apple firmware while on the same machine Rockbox can't cope with resized mpeg-1 witch low bitrate - i mean it's playing it but with low FPS.
I guess iPod must have some hardware accelerated mpeg4 decoder but Rockbox programmes don't know how to access it.
Probably (I am not a programmer, I have no idea, maybe I am wrong) iPod also is using dedicated co-processor to decode mp3 and aac music and because of this it is consuming less energy than Rockbox that is using main universal CPU.
You can forget about watching any movie on Rockbox - any of those little processors will never handle any video format in fair FPS.

But hey, do you want to listen to the music or watching movies on microscopical screen? ; )

[robin0800]

I guess iPod must have some hardware accelerated mpeg4 decoder but Rockbox programmes don't know how to access it.

Yes the ipods have

Analog Devices ADV7179

and some

Broadcom BCM2722

video chips see http://www.rockbox.org/wiki/IpodStatus for details
but the bottom line is that there is no documentation for these chips

[sirdan]

Ipod video is slow because it was tested with 320x240 full screen video right? What about 240x176/128 watching in horizontal position?

But hey, do you want to listen to the music or watching movies on microscopical screen? ; )

Heh i dont watch almost at all, probably others do. I just wanted to raise again the discussion. Maybe there is a developer that feel strong on video codecs field Well generally we have many audio codecs, can the audio quality be improved? Thats priority of course but what esle can be done in RB? Mpeg4 playback would be nice addon.

[soap]

Probably (I am not a programmer, I have no idea, maybe I am wrong) iPod also is using dedicated co-processor to decode mp3 and aac music and because of this it is consuming less energy than Rockbox that is using main universal CPU.

1 - No.   No dedicated mp3 and aac coprocessor.
2 - Rockbox BEATS Apple firmware for MP3 playback.

You can forget about watching any movie on Rockbox - any of those little processors will never handle any video format in fair FPS.

Odd, Rockbox plays back video on the Color/Photo and Nano at fine FPSs.

[AlexP]

On everything except the ipod video, Rockbox gets better frame rates than the original firmware (where they do video at all that is).

[sirdan]

On everything except the ipod video, Rockbox gets better frame rates than the original firmware (where they do video at all that is).

I think that problem can be resolved for ipod video users.

Since Im keen on video compression/decompression stuff, i decided to do some tests showing how encoding settings influence playback. Im on sansa e200 but it can be applied to others targest as well. Here are my benches on Elephant dream:

240x128; 24fps; 11min realtime playback
240x128; 24fps; 7min bench time
240x128; 15fps; 5min bench time
240x128; 12fps; 4min bench time
176x100; 24fps; 5min bench time
240x128; 24fps; half bitrate; 6:30 min bench

Every file has the same duration, so we see how playback speed scales with different configs. Shorter bench time=battery savings.

Now word about Ipod video. It plays 320x240 24fps clip at 19 fps. So the most simple way to achieve realtime is convert original fps to below 19 fps..
Removing frames goes to jerky playback. They will be blended to retain smoothness. Blending will blurry image but not much.
First make avisynth script with dgindexed video source or loaded by dshowsource. Then add at the end of the script

Code: [Select]

ConvertFPS(18). AVScript (on windows only) can be used as input for mencoder.exe to produce 18 fps smooth file. All mencoder builds encode mpeg with only 12,15,24,25,30..etc - dvd compatible framerates. It should be patched probably with

Code: [Select]

c->strict_std_compliance = FF_COMPLIANCE_INOFFICIAL; to remove that limitation. I havent reached that point as i dont know how compile mencoder at now. If anybody is that freaky to make all steps, he should be happy with his videos on ipod video.

[saratoga]

On everything except the ipod video, Rockbox gets better frame rates than the original firmware (where they do video at all that is).

I think that problem can be resolved for ipod video users.

Since Im keen on video compression/decompression stuff, i decided to do some tests showing how encoding settings influence playback. Im on sansa e200 but it can be applied to others targest as well. Here are my benches on Elephant dream:

240x128; 24fps; 11min realtime playback
240x128; 24fps; 7min bench time
240x128; 15fps; 5min bench time
240x128; 12fps; 4min bench time
176x100; 24fps; 5min bench time
240x128; 24fps; half bitrate; 6:30 min bench

Every file has the same duration, so we see how playback speed scales with different configs. Shorter bench time=battery savings.

Now word about Ipod video. It plays 320x240 24fps clip at 19 fps. So the most simple way to achieve realtime is convert original fps to below 19 fps..

I think the wiki already recommends that people not use higher frame rates then they can decode, so its not really necessary that you "resolve" this problem.

[Llorean]

Not only does the wiki recommend this, but most of the presets and recommended conversion command lines predate performance improvements, so some of them (notably the WinFF ones) should result in files converted to framerates low enough to even leave some elbow room for particularly complex scenes and bitrate spikes.

[reszerve]

I take it there's no progress on this? 

I realise that many targets won't be capable, but as a gigabeat owner I find it wretched having to convert to mpegs.  This is my second most wanted improvement for RockBox.

Going forward, even those awful locked-down players like Zune seem to have this feature now.

[Llorean]

Even with a gigabeat you'd in many cases still have to convert your files anyway, because they'd need to be adjusted for the screen size.

If it's the requirement for conversion that's wretched, don't expect that ever to realistically be banished for many players. Even the Zune has format restrictions and size restrictions, and often the file needs to be converted while syncing.

[AlexP]

If someone were to write the code to decode other codecs, then I'm sure it'd be incorporated.  However, converting videos is a damn site quicker and easier than writing e.g. xvid support for Rockbox.  Someone that wants it enough to bother needs to write it.