Currently Audacity uses a 64k input buffer when reading mp3s with libmad. This has worked great for the most part, but I've received a report that one particular mp3 will put Audacity into an infinite loop upon import. Since a trace revealed that the input callback was looping endlessly, I suspected that the buffer wasn't big enough to hold an entire frame, and the input callback was sending libmad the same partial frame over and over.
I suggested that this user double the input buffer size, and that fixed the problem.
Earlier when I asked you about this issue, you said mp3 frames are relatively small (around 418 bytes) so I'm very surprised that 64k wasn't big enough to hold an entire frame. Can you think of any other reason why increasing the buffer size would solve this problem?
Is there a maximum size for mp3 frames that I could safely set as the buffer size, or would I have to dynamically size the buffer to be completely safe?
Thanks, Joshua
Joshua Haberman wrote:
Currently Audacity uses a 64k input buffer when reading mp3s with libmad. This has worked great for the most part, but I've received a report that one particular mp3 will put Audacity into an infinite loop upon import. Since a trace revealed that the input callback was looping endlessly, I suspected that the buffer wasn't big enough to hold an entire frame, and the input callback was sending libmad the same partial frame over and over.
This sounds like the problem I encountered. The final frame of the mp3 file is truncated, so my input buffering routine returns that which mad_header_decode() hasn't consumed -- the truncated frame. mad_header_decode() indicates the partial frame held in the buffer isn't enough to decode via returning MAD_ERROR_BUFLEN which normally means it wants more input. And normally the partial frame would be copied from the tail of the input buffer to the buffer head and would be followed by the continuation of the frame from the mp3 file. But in this error case we are at the end of the file. And the process loops until otherwise terminated.
I found two ways to work around this nit. One is to pad out the buffer with enough data to hold the balance of the worst case truncated frame. Or have your decoder recognize two consecutive returns from mad_header_decode() where MAD_ERROR_BUFLEN is the return value at the same file offset and terminate the process at that point.
I suggested that this user double the input buffer size, and that fixed the problem.
That shouldn't have remedied the problem I describe above.
Earlier when I asked you about this issue, you said mp3 frames are relatively small (around 418 bytes) so I'm very surprised that 64k wasn't big enough to hold an entire frame. Can you think of any other reason why increasing the buffer size would solve this problem?
If you were seeing this problem because of a corrupt final frame, it is possible the input routine is incorrectly reporting the true (corrupt) data size for some reason in the case of the larger buffer.
Is there a maximum size for mp3 frames that I could safely set as the buffer size, or would I have to dynamically size the buffer to be completely safe?
Except for this nit, mad_header_decode() should work with as small as a 418 byte (+ 8 byte pad) buffer. All it wants is enough room to guarantee it wont over-read the buffer memory in the process of decoding the frame irrespective of what type of frame (valid or corrupt) is thrown at it.
-john
Joshua Haberman wrote:
Currently Audacity uses a 64k input buffer when reading mp3s with libmad. This has worked great for the most part, but I've received a report that one particular mp3 will put Audacity into an infinite loop upon import. Since a trace revealed that the input callback was looping endlessly, I suspected that the buffer wasn't big enough to hold an entire frame, and the input callback was sending libmad the same partial frame over and over.
I suggested that this user double the input buffer size, and that fixed the problem.
Earlier when I asked you about this issue, you said mp3 frames are relatively small (around 418 bytes) so I'm very surprised that 64k wasn't big enough to hold an entire frame. Can you think of any other reason why increasing the buffer size would solve this problem?
Sounds like it could be a bug in your input/streaming code, triggered by the unlucky buffer size? Out of curiosity, what bitrate and sampling frequency was the mp3 that caused problems?
Is there a maximum size for mp3 frames that I could safely set as the buffer size, or would I have to dynamically size the buffer to be completely safe?
The absolute theoretical maximum frame size is 2881 bytes: MPEG 2.5 Layer II, 8000 Hz @ 160 kbps, with a padding slot. (Such a frame is unlikely, but it was a useful exercise to compute all possible frame sizes.) Add to this an 8 byte MAD_BUFFER_GUARD, and the minimum buffer size you should be streaming to libmad in the general case is 2889 bytes.
Theoretical frame sizes for Layer III range from 24 to 1441 bytes, but there is a "soft" limit imposed by the standard of 960 bytes. Nonetheless MAD can decode frames of any size as long as they fit entirely in the buffer you pass, not including the MAD_BUFFER_GUARD bytes.
A 64K buffer is surely large enough, which makes me think you have a different kind of problem...
* Rob Leslie (rob@mars.org) wrote:
Joshua Haberman wrote:
Currently Audacity uses a 64k input buffer when reading mp3s with libmad. This has worked great for the most part, but I've received a report that one particular mp3 will put Audacity into an infinite loop upon import. Since a trace revealed that the input callback was looping endlessly, I suspected that the buffer wasn't big enough to hold an entire frame, and the input callback was sending libmad the same partial frame over and over.
I suggested that this user double the input buffer size, and that fixed the problem.
Earlier when I asked you about this issue, you said mp3 frames are relatively small (around 418 bytes) so I'm very surprised that 64k wasn't big enough to hold an entire frame. Can you think of any other reason why increasing the buffer size would solve this problem?
Sounds like it could be a bug in your input/streaming code, triggered by the unlucky buffer size?
You're right, it was a bug in my input callback. I've looked over that routine several times but I missed the problem up until now. Though I was preserving the data left over from the previous frame, I passed to mad_stream_buffer() only the length of the data I had just read from the file, instead of adding that length to the length of the leftover data. It was a subtle bug that rarely manifested itself.
We seem to have come across another instance of strange behaviour dealing with libmad. Quoting Dominic Mazzoni, Audacity's main author:
All this time I thought that libmad was just slow compared to xaudio, but I just realized that importing a 1-minute song takes exactly 1 minute, importing an 11-second song takes exactly 11 seconds, and importing a 3:04-minute song takes exactly 3:04.
BTW, I'm doing this on an Athlon/1.2 GHz, so it should be decoding at something like 10x real time.
Any idea what this would be?? Does libmad have built into it internal timekeeping that would cause this kind of behaviour?
Thanks much, Joshua
* To mad-dev@lists.mars.org (joshua@haberman.com) wrote:
We seem to have come across another instance of strange behaviour dealing with libmad. Quoting Dominic Mazzoni, Audacity's main author:
Scratch that, it was our code again. I should have known, but the extreme coincidance that files took exactly as long to open as the song length seemed too strange to be accidental.
Joshua
Dear Sir,
Is there a maximum size for mp3 frames that I could safely set as the buffer size?
According to "Bertrand Petit" who wrote madlld (mad low-level demonstration): (can found in MAD main page)
* When this occurs, the remaining unused bytes must be * put back at the beginning of the buffer and taken in * account before refilling the buffer. This means that * the input buffer must be large enough to hold a whole * frame at the highest observable bit-rate (currently 448 * kb/s). XXX=XXX Is 2016 bytes the size of the largest * frame? (448000*(1152/32000))/8
Yick Hong Kong, China
joshua@haberman.com wrote:
Currently Audacity uses a 64k input buffer when reading mp3s with libmad. This has worked great for the most part, but I've received a report that one particular mp3 will put Audacity into an infinite loop upon import. Since a trace revealed that the input callback was looping endlessly, I suspected that the buffer wasn't big enough to hold an entire frame, and the input callback was sending libmad the same partial frame over and over.
I suggested that this user double the input buffer size, and that fixed the problem.
Earlier when I asked you about this issue, you said mp3 frames are relatively small (around 418 bytes) so I'm very surprised that 64k wasn't big enough to hold an entire frame. Can you think of any other reason why increasing the buffer size would solve this problem?
Is there a maximum size for mp3 frames that I could safely set as the buffer size, or would I have to dynamically size the buffer to be completely safe?
Thanks, Joshua
Lam Yick Yan superylam@netscape.net wrote:
Is there a maximum size for mp3 frames that I could safely set as the buffer size?
According to "Bertrand Petit" who wrote madlld (mad low-level demonstration): (can found in MAD main page)
- When this occurs, the remaining unused bytes must be
- put back at the beginning of the buffer and taken in
- account before refilling the buffer. This means that
- the input buffer must be large enough to hold a whole
- frame at the highest observable bit-rate (currently 448
- kb/s). XXX=XXX Is 2016 bytes the size of the largest
- frame? (448000*(1152/32000))/8
I'm not sure where 448 kb/sec comes from.... ?
If you substitute 320 kb/sec into the above formula however, the largest possible MP3 frame comes out as 1440 bytes which I believe is the correct answer.
Andre --
__________________________________________________ Do You Yahoo!? Everything you'll ever need on one web page from News and Sport to Email and Music Charts http://uk.my.yahoo.com
Andre wrote:
According to "Bertrand Petit" who wrote madlld (mad low-level demonstration): (can found in MAD main page)
- When this occurs, the remaining unused bytes must be
- put back at the beginning of the buffer and taken in
- account before refilling the buffer. This means that
- the input buffer must be large enough to hold a whole
- frame at the highest observable bit-rate (currently 448
- kb/s). XXX=XXX Is 2016 bytes the size of the largest
- frame? (448000*(1152/32000))/8
I'm not sure where 448 kb/sec comes from.... ?
448 kbps is the highest possible bitrate for MPEG audio, but it is only possible for Layer I. Since there are only 384 samples per frame in Layer I, the above formula doesn't calculate the correct frame size.
If you substitute 320 kb/sec into the above formula however, the largest possible MP3 frame comes out as 1440 bytes which I believe is the correct answer.
This is correct for MPEG-1 and MPEG 2.5 Layer III. In theory frames at 320 kbps could have their padding slot bit set for a total of 1441 bytes.
-rob