A second thought:
It might be possible to apply a "diffusion" to the quantized frequencies in the decoder, sort of like dither, but with the effect of treating all values within some window of the quantized value equally likely. The window could be larger for values quantized with fewer bits, and smaller for values with more bits.
It might make the sound worse; I don't know.
Maybe this is what you were thinking of, Gabriel? Is it worth trying?
-rob