WARNING: current CVS recording code not stable [ NOW FIXED! ]

[Davide-NYC]

I have several loooong recordings of FM Radio I did for a stability test of the new recording code.
I am not willing to listen to hours upon hours of (boring) FM radio.
I have scanned the waveform visually using Audacity and find no drop outs.

Question: Is there a program that will find contiguous zero samples? Like if 10 or more samples are zero then inform the user somehow. I'd like to scan my test files for dropped samples, but want to do it in a semi-automated fashion.

Please let me know...

[petur]

The missing samples are not shown as zero samples. They are just missing. As if you would have cut out a part of them (like in Audacity)

My 1:44 hour recording has 3 glitches, two of them are even more weird as last time: a part of the recording is repeated before it moves on to where it should have been.

[Davide-NYC]

So the only way for me to test this is to listen?  

[jhMikeS]

AHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH

I'm a bit frustrated and fear the problem lies in some obscure bug not nescessarilty directly in the recording code that will take some mega effort to track down. There was a symptom on the H120 where having the callback pointer in IRAM would cause a freeze within about 15s. Something else could be overwriting adacent memory locations. I'm not dicounting the recording code itself but I just can't find a problem and I usually find them readily. Be mindful that the recording update revealed a problem with handling memory throughout the entire system...not with recording itself. So where to move next and how to proceded I'm not really certain but really want you guys to have it working perfectly.

It not a repeat since the first has a snare fill and the second doesn't and the instruments are playing differently for each section.

[jhMikeS]

Ok, right now I'm just gonna let radio record for a couple hours and see.

[jhMikeS]

Just got done listening to 2h 14m 25s of flawless wav recording with a voice file present and AGC safety clip. Voice restarted in the menus as expected and no crash.

Thought I had a glitch at 1:09:23 but it turned out it was just the words f***in nothin' censored in a odd way (Metallica). F***in nothin' about summs it up though.

Petur: I can send you the 1.32GB file to prove it if you like. Have you considered isolating your DAP from vibration? I don't doubt the recording environment must be kinda loud. Just a thought.

What I hear in your recording is exactly the sort of thing that should happen if it writing is blocked and the output fills: it _must_ drop data since it has nowhere to put it.

I suggest a lengthy recording in a quiet envoronment like just recording radio with your usual setup. If that is ok, try recording with the DAP and speakers in a box and cranking up the volume to a concert level with the corresponding low end. See what happens.

These long recordings mean a lot of trips around all the buffers and you got five glitches and I got none. I would expect something to show up for me if the recording itself were faulty but I'll keep an open mind. This problem could be unique to the H300 series but that would rule out recording itself since aside from SPDIF the code is identical to yours.

The crashes at stop are certainly another issue (since all data is written including the headers which are the very last thing updated before closing a file) as is the AGC crash (DivX0) reported elsewhere.

Sorry for being repetitve but I want my analysis well documented.  

[mlind]

How about recording a sine wave or something equally constant?
This way a glitch will probably be clearly visible.

How long is the drop-out? A few samples or a few seconds?

A serious test would be to record with the DAP lying still for a period of time, and then be moving in some disturbing way for an equal amount of time, and so on back and forth with the different circumstances well documented for comparison with the recorded result...

[jhMikeS]

The skips petur had are exactly on chunk (2048 sample) boundaries and clearly discernable, both by ear and in my audio editor at the sample level. Any skips in the radio recording would be too but nothing. The DAP was still for the entire time when I tested. Inability to flush data will drop the latest data and keep the codec write position from overrunning the read (where disk writing start) position. Not preventing that would just crash it and destroy the markers for file starts and ends.

EDIT: I should also mention that all skips were forward in time, not random or backwards which is the correct behavior for the inability to write to disk for some reason recording would have no control over.

[petur]

except for the last two where 2 seconds recording was saved twice...

fyi, I've done many recordings (over a year time) in the same conditions or even worse (too load, too much bass so more vibrations). Never had a glitch. The problems started after the new recording framework was committed though the first recordings were fine.

My bet is more on disk fragmentation. I've defragged now and will try to keep it that way.

[pabouk]

Inability to flush data will drop the latest data and keep the codec write position from overrunning the read (where disk writing start) position.

Just a little idea: Could not the buffer dropping event be logged using logf to be able  to narrow down possible reasons of the glitches?

[pupil]

just a quick question, been following the problems you're having with recording recently. What is the last dated CVS build that you would recommend installing if (like me) you are wanting perfect recordings without these reported glitches.

thanks.

[jhMikeS]

Just a little idea: Could not the buffer dropping event be logged using logf to be able  to narrow down possible reasons of the glitches?

It already does.

EDIT: And I've never seen the message logged even running WavPack at 88.2 which requires a thread priority boost (in addition to CPU boost) to flush. The prio boost causes almore instant completion of data writing.

[jhMikeS]

just a quick question, been following the problems you're having with recording recently. What is the last dated CVS build that you would recommend installing if (like me) you are wanting perfect recordings without these reported glitches.

thanks.

I don't think you'll have any problem with the latest. As I said, I did a concert length recording on my H120 and had not a single skip.

The only possible output buffer full glitch would be from the file apis being unable to write. There is no issue with any encoders being able to keep up with the data on the input end. WAV and AIFF never boost at any samplerate, even 88.2 kHz.

[petur]

just found a way to make it freeze:

from the rec screen:
- pres rec to start recording
- press stop
- press rec again while still flushing
*bingo*

[jhMikeS]

I can't get it to do it (freeze permanently that is). It just sticks for a second or two then just stops recording as it should. These are just the sort of operations I performed when preparing the update but will look further. The recording screen probably isn't quite set up for asynchronous stops. The H120 has no RTC and create_datetime_filename might have something to do with it since it waits until the next second before returning the file name. My x5 is in the shop so I can't test this with the RTC .

While stopping, the state is still "recording" so the record button should be requesting a file split.  The split should be rejected in audio_new_file because the stop should be waited for first and you can't do audio_new_file if not already recording.