Greetings to all,
I'm a new user to MAD and it's immediately become my standard for mp3 listening and decoding. The first thing that shook me when using Mad, was something that I never had encountered before, the mp3 clipping indicator. Low-bit distortion is one thing, but high-bit problems is surely a worse one, if not the worst! And when I witnessed Mad reports clipping even with files produced by the best encoders like Lame, this rang a red alert for me: are we constantly exposed to clipping distortion when using winamp's decoder or other decoding apps?
Research in the web showed that there are people who claim that clipping is a well-known artifact of lossy mp3 compression. For me this is still a mystery, since I had the misconception (or is it true ?) that a well-established or even commercial encoder cannot possibly produce an mp3 that exceeds the 100% level. I am not familiar with the binary pcm structure of wav files, but I know it uses 16-bit numbers, so clipping is by the definition either the presence of a 17th bit (impossible) or the miscoding of a music level of 99% as 100% (which means that everything above 99% is clipped cause it's represented as a constant 100%). On the other hand, I understand that an encoded mp3 level of 99% is possible to be interpreted by a decoder as a level of 100%, which would mean that clipping is indeed possible to occur, but *only* when the file is badly decoded.
For me two possibilities exist:
a) The offending mp3 does not contain clipped samples but Mad misreports them (rather improbable, I do trust Mad's author programming skills)
b) The offending mp3 does contain clipped samples which were produced by bad encoding (therefore Mad is correct)
c) The offending mp3 does not contain clipped samples, but a decoder (either because of bad programming or because of the nature of mp3 lossy compression) *could* misinterpret them, so a decoder should take corrective action (i.e. attenuation like Mad does).
I cannot easily find a way to produce convincing evidence as to which of the above is the case. At first sight one could propose: "just take an mp3 and decode it with a standard decoder and see if it contains clipped samples". However this is not good enough for me, for the following reason: If i use 10 standard decoders I'll get 10 wavs which will almost certainly differ. The next step would be to *visually* inspect the wavs for clipping, cause I do not think that any wav editor is capable of detecting and reporting (and possibly take action like Mad's attenuation) levels above 100% when opening a wav or importing an mp3 (am I wrong here?).
In conclusion, to rephrase my (naive ?) questions: how is it in principle possible that Mad detects clipping, how does Mad do it and where exaclty is Mad detecting it: a) is clipping stored inside the mp3 file content? b) is it an artifact of decoding programs ?
In addition, can you suggest another reliable means of verifying that an mp3 contain or not clipped content ?
Excuse me for the lenghty post, but I think the issue I bring forward is an important one. For one, I would hate the idea of being forced to first normalise all my wav's to a safe e.g. 95% before encoding it, to just prevent the eventual clipping upon decoding.... Moreover if my worries are justified, we should definitely propose "response to mp3 clipping" as a very important criterium for the evaluation of mp3 decoders (and encoders ?) in that famous www page that does "objective" mp3 software comparisons.
Best regards, Mits
"D D" wrote:
I'm a new user to MAD and it's immediately become my standard for mp3 listening and decoding. The first thing that shook me when using Mad, was something that I never had encountered before, the mp3 clipping indicator. Low-bit distortion is one thing, but high-bit problems is surely a worse one, if not the worst! And when I witnessed Mad reports clipping even with files produced by the best encoders like Lame, this rang a red alert for me: are we constantly exposed to clipping distortion when using winamp's decoder or other decoding apps?
In a word, probably.
[...] In conclusion, to rephrase my (naive ?) questions: how is it in principle possible that Mad detects clipping, how does Mad do it and where exaclty is Mad detecting it: a) is clipping stored inside the mp3 file content? b) is it an artifact of decoding programs ?
To understand the clipping issue, it's helpful to understand a little bit about the MP3 decoding process. There are two steps relevant to this discussion.
+---------+ 01011001100010 ---> | decoder | ---> +0.298238, +0.019343, -0.374920, ... MP3 data +---------+ PCM samples
The PCM samples produced from decoding are relative to a "full scale" range of -1 to +1. In other words, these samples are fractional numbers with absolute value (usually) less than 1.
As will be explained, no clipping occurs here, although these values can sometimes go beyond full scale.
+--------------------+ PCM samples ---> | scaling, rounding, | ---> 9773, 634, -12285, ... | dithering | 16-bit output +--------------------+
To get a 16-bit (or 8-bit or 24-bit) output value from a PCM sample, you need to scale the number -- for example, multiply by 2^15. Thus the full 16-bit integer range -32768 to +32767 is used to represent the fractional numbers between -1 and +1.
A problem arises when the decoder produces PCM samples beyond full scale, that is, with absolute value greater than 1. This can happen even in fully compliant decoders, and is not (necessarily) the result of a programming error. It happens due to the MP3 algorithm itself and the encoding methods applied to the original signal.
Encoders *could* try to minimize this occurrence, but my guess is most are probably unaware of the effect or do not pay very close attention to samples near full scale, so these samples can sometimes creep above full scale depending on the level of quantization. Encoders also work in the frequency domain, so it is probably difficult to realize the effect on samples in the time domain without following up with a full decode.
In any case, decoded samples beyond full scale are a fact of life. The question is, what to do about it? These can't be represented in normal output using the scaling method I just described, so something must be done.
The simplest answer is to clip the signal, and this is probably what 95% of all decoders do. Any sample above full scale is treated the same as full scale. This adds distortion to the output.
Another answer is to use a limiting/compressing technique on samples near and beyond full scale. This has been discussed on this list and is something I've considered and may implement at some point.
In the case of the current MAD plug-in, clipping can be used in a feedback loop to attenuate the signal so that further clipping can be avoided.
It's important to keep in mind the two distinct steps I described to get to the final output. Some decoders combine these steps, but this limits their flexibility. Strictly speaking, "MAD" only performs the decoding step; the issue of clipping is outside its scope. Applications that use MAD, like the plug-in for Winamp, are responsible for the policies that affect things like clipping and dithering. It's a fine distinction, but other apps that use MAD may implement different policies.
In addition, can you suggest another reliable means of verifying that an mp3 contain or not clipped content ?
Since most decoders combine the decoding step with the scaling and clipping step, it's impossible to know from their output how much clipping has occurred. It's the same as trying to determine the number of clipped samples in a 16-bit WAVE file: you can guess that the samples at -32768 and +32767 were clipped from some greater amplitude, but this isn't always the case. That's why tools like Cool Edit only report "possibly clipped samples."
Recent versions of the command-line `madplay' will report the number of clipped samples and peak amplitude, but this is the same information the plug-in gathers when attenuation is disabled, and since it uses the same MAD decoder back-end, this may not be the kind of verification you're looking for. :-)
The sample MPEG audio decoder implementation in the ISO dist10 source code floating around actually reports clipped samples as it's decoding. It doesn't give a total though, and you'd probably have to compile it yourself, so I don't know how useful this is.
Hope this helps.
At 06:12 PM 2/13/01 +0000, D D wrote:
I'm a new user to MAD and it's immediately become my standard for mp3 listening and decoding. The first thing that shook me when using Mad, was something that I never had encountered before, the mp3 clipping indicator. Low-bit distortion is one thing, but high-bit problems is surely a worse one, if not the worst! And when I witnessed Mad reports clipping even with files produced by the best encoders like Lame, this rang a red alert for me: are we constantly exposed to clipping distortion when using winamp's decoder or other decoding apps?
Yes, and the more recent the music, the more likely it is that your average mp3 is clipping, since record producers these days seem intent on compressing the life out of their recordings to increase average volume. Yuck. The mp3 encoding process can add even more clipping, making a bad situation worse. The best solution is to normalize to less than full scale *before* encoding. See my page:
http://fastforward.iwarp.com/norm.html
for a discussion on the audibility of clipping introduced by encoding to mp3.
ff123
this is what I assume:
in downsampling from whatever medium the recording was made (24-bit, 96Khz, for example), to keep the dynamic range of the recording at it's fullest, whoever masters the CD will normalise the audio to peak of the 16-bit range. I would hope that even though samples exist at the edges of the range (reported as "possibly clipped samples" by cool edit pro), the true shape of the waveform is not clipped, but just touches the amplitude limit.
are you suggesting is that the encoding process (in some/most cases) furthar distorts the waveform so that it is clipped beyond the max amplitude of the 16-bit range?
Matt
... ah decided to check out your page before pressing send.
I now see how clipping when normailizing to peak *can* occur.
ff123 wrote:
At 06:12 PM 2/13/01 +0000, D D wrote:
I'm a new user to MAD and it's immediately become my standard for mp3 listening and decoding. The first thing that shook me when using Mad, was something that I never had encountered before, the mp3 clipping indicator. Low-bit distortion is one thing, but high-bit problems is surely a worse one, if not the worst! And when I witnessed Mad reports clipping even with files produced by the best encoders like Lame, this rang a red alert for me: are we constantly exposed to clipping distortion when using winamp's decoder or other decoding apps?
Yes, and the more recent the music, the more likely it is that your average mp3 is clipping, since record producers these days seem intent on compressing the life out of their recordings to increase average volume. Yuck. The mp3 encoding process can add even more clipping, making a bad situation worse. The best solution is to normalize to less than full scale *before* encoding. See my page:
http://fastforward.iwarp.com/norm.html
for a discussion on the audibility of clipping introduced by encoding to mp3.
ff123
-
D D -
ff123 -
Matt Makauskas -
Rob Leslie