My idea is to use an adaptative dithering in frequency space. As in decoding we know how much bits were used in a sfb, we could perhaps use this data to adjust the amount of dithering. This way we could dither more when less bits were used, and dither less when more bits were used.
Hmm, an interesting idea.
In another forum I've tried to make the distinction between quantization noise introduced by the encoder (as part of the lossy psychoacoustic coding algorithm) and quantization noise introduced by the decoder (in the output PCM samples.) Dithering the output PCM only affects the latter.
I've been under the assumption the decoder has no control over the quantization noise introduced by the encoder. After all, the quantized bits are gone, so it's not possible to know what the error was, even knowing the number of bits in the quantized result. I'm not sure how you would apply dither in this case since there is nothing to dither from.
Perhaps frequency dithering could take place instead in the encoder, where the quantization error is known?
Other thoughts?
-rob