Discussions (& strategies) on getting WMA and M4A (AAC) decoding in real-time.

[Davide-NYC]

I wish you (JonathanHull) all the luck in the world circumventing the hardware/software chokehold your legitimately aquired music is in. DRM should be harware/software independant. I like the idea of individually encoding the buyers data into the audio itself. "bitprinting" every file sold with the buyer's name and purchase order number and then distributing the WAV files in a lossless format is the only way I would ever consider 'joining in'. 224kbps? No thanks.

But I wouldn't want this thread to get ahead of itself, so back on topic!  

The idea here is to discuss methods for better coordinating the optimization effort for these two decoders. In other words: How can we direct volunteer developer hours in a more efficient manner? It would be a crying shame if two ASM capable devs works on the same hunks of code at the same time. This would be a tragic waste of (wo)man hours.

I'm going to pose this question on the developer mailing list soon, but I'd like to let it 'mature' here first so that it doesn't come across half baked.

I think a list of files with a brief description of what the file does, followed by the dev willing to look at the file, followed by some comments and then the estimated state of optimization. (maybe boost ratios for different bitrates and targets?)

These are the files in the libwma folder from the flyspray patch:

asf.c
avcodec.h
avformat.h
avi.h
avio.c
avio.h
aviobuf.c
codeccontext.h
common.h
cutils.c
dsputil.h
dtypes.h
file.c
fixedpoint.c
fixedpoint.h
futils.c
parser.c
test.c
utils.c
wmadeci.c
wmadeci.h
wmadeci_mixed.c
wmadeci_mixed.h

What do these files do exactly? What do they need done to them? Do we need them all for wma playback? (asf.c and avi.h for example) etc etc etc.

[JonathanHull]

Oh man do I agree.

These songs were all bought on iTunes 2+ years ago with gift certificates from family. I would not, and will not, but DRM'd music. But I figure if I own some might as well try to listen to it.

I also have some WMA files I stripped the DRM out of that came from Napster (also with a gift certificat), but thats another story.

[Lear]

The idea here is to discuss methods for better coordinating the optimization effort for these two decoders. In other words: How can we direct volunteer developer hours in a more efficient manner? It would be a crying shame if two ASM capable devs works on the same hunks of code at the same time. This would be a tragic waste of (wo)man hours.

This hasn't really been a problem, as far as I know. The number of developers working on the codecs has been quite small. Some coordination, if needed, occurs on IRC.

I think a list of files with a brief description of what the file does, followed by the dev willing to look at the file, followed by some comments and then the estimated state of optimization. (maybe boost ratios for different bitrates and targets?)

I not so sure a list would be very useful... Before even considering assembler optimization, you need to know what parts to optimize. To know that, you first need to have a working codec, perhaps with some basic/obvious optimizations applied (typically a few small inlined functions with target-specific code).

Then you do a profile run and see where the most time is spent. At that point it is a good idea to try to figure out what buffers/constant tables to put in the fast internal RAM, and move them there. This typically involves changes in several files, and is best done by a single developer.

Once you've done those basic steps (which is about the state the AAC decoder is at), you are at a point where a list might help. But it often turns out that most of the processing is done in a small number of functions, involving maybe 2-3 files.

[Davide-NYC]

Then you do a profile run and see where the most time is spent.

Is there any way for me to do some testing and then report my findings here? I am willing to encode whatever audio to all possible bitrates in both AAC and WMA and do whatever testing is necessary.

It is this type of time consuming 'lesser' skilled task that I can contribute to the effort.
If this is at all possible I will need explicit instructions.

 

[Lear]

Is there any way for me to do some testing and then report my findings here? I am willing to encode whatever audio to all possible bitrates in both AAC and WMA and do whatever testing is necessary.

Not really, I'm afraid. What could possibly help is if you find a file that doesn't play real-time when expected, or need an unusually high boost ratio, given the bitrate and type of encoding. That is a bit premature in the case of AAC though, since there are some files that won't play back at all at the moment.

[saratoga]

Then you do a profile run and see where the most time is spent.

Is there any way for me to do some testing and then report my findings here? I am willing to encode whatever audio to all possible bitrates in both AAC and WMA and do whatever testing is necessary.

I'm not sure how useful it would be, but you're welcome to compile a build with profiling enabled, and post the results here.  I'd guess any interested developers probably did this first thing since its not that hard and you can't really optimize without doing it, but if you're interested . . .

[Davide-NYC]

I got a compile error when trying to compile a build with profiling enabled.

I thought it might be the fact that I was using gcc 3.4.4 instead of 3.4.6 (cygwin)

In my newbie attempt to update my gcc I totally bunked my cygwin environment.

Haha.

Will do a clean cygwin install, but I'll be back to gcc 3.4.4 (which may not even have been the problem) and no way to make a build with profiling enabled.

Can someone help? If so how?

[bluebrother]

gcc 3.4.4 has an error. This has been discussed on rockbox-dev (and on irv) recently. Please:
- make sure you can build without profiling properly first
- use gcc 3.4.6 (for m68k)
- when asking for help provide some more details. Currently I can only guess why building failed for you (though I'm pretty sure gcc was the first cause, but this doesn't imply it's the only one)
- discuss build errors in the compilation forums. This isn't exactly the right forum as the topic moved.
Also, I'd suggest you trying the vmware image. Afaik building is a lot faster than in cygwin and it's completely preconfigured.

[Davide-NYC]

OK, I'm using the VMWare image (gcc 3.4.6) and I still get an error:

LD aac.elf
/opt/m68k/lib/gcc/m68k-elf/3.4.5/../../../../m68k-elf/bin/ld: region PLUGIN_IRAM
is full (/home/Administrator/rockbox-devel/build.profiling/apps/codecs/aac.elf
section .ibss)
collect2: ld returned 1 exit status
make[2]: *** [/home/Administrator/rockbox-devel/build.profiling/apps/codecs/aac.
elf] Error 1
make[1]: *** [build-codecs] Error 2
make: *** [all] Error 2

Going to IRC now to see if anyone can help me.

Linuxstb has informed me that this error means that the AAC code is simply too big with profiling enabled. I am not the person to 'shrink' it so I guess I have reached an impasse with this particular method of contributing.

[Lear]

Hm... That's annoying. The profiling does put some code in iram. Only way around that is to slightly reduce the amount of iram used by normal data.

Some less important data that can definitely be removed; it was mainly added because it wasn't much and there was room for it. Not sure if it is enough to allow a profiling build though.

[Lear]

Gah! 4 bytes short! On coldfire at least; haven't tried arm. Will remove some 200 bytes though.

[Davide-NYC]

OK so now it builds with proifiling enabled. Thanks!
Of course I do not get a profile.out file in the root of my player.

I do not undertstand the SourceProfiling wiki page 100%.
What exactly do I have to put in the source and where should it go?
Or even make me a patch and tell me what to run and what to look for.

 

[Lear]

OK so now it builds with proifiling enabled. Thanks!
Of course I do not get a profile.out file in the root of my player.

Correct. You need to edit aac.c a little first.

I do not undertstand the SourceProfiling wiki page 100%.
What exactly do I have to put in the source and where should it go?
Or even make me a patch and tell me what to run and what to look for.

Add the following to the beginning of the codec_start function (but after the "rb = api;" line):

Code: [Select]

   #ifdef RB_PROFILE
    rb->profile_thread();
    #endif

Then add the following at the end of the same function, before the "return" line:

Code: [Select]

   #ifdef RB_PROFILE
    rb->profstop();
    #endif

Then you should get the profile.out file.

[Davide-NYC]

If these are #ifdef'ed then why are they not in the source all the time?
Please explain.

[pabouk]

Because you have to decide which parts of the code you are going to profile.