Hello,
In the past while I have been rounding up development for a streaming server for the radio station I am part of. However, during the last stress testing phase I noticed that memory usage slowly increases as streaming clients disconnect and connect. At first I was convinced it was some leak in my own code but valgrind didn't confirm that this was happening. However, I noticed that there was quite some lost memory originating in the mad_layer_III() function in layer3.c:
==5957== 7,701 bytes in 3 blocks are definitely lost in loss record 68 of 85 ==5957== at 0x481B72C: malloc (in /usr/lib/valgrind/x86-linux/vgpreload_memcheck.so) ==5957== by 0x485F1C1: mad_layer_III (in /usr/lib/libmad.so.0.2.1) ==5957== by 0x485429A: mad_frame_decode (in /usr/lib/libmad.so.0.2.1) ==5957== by 0x7775: decode_data (server.c:318) ==5957== by 0x9387: server (server.c:980) ==5957== by 0x4874164: pthread_start_thread (in /lib/libpthread-0.10.so) ==5957== by 0x4A176E9: clone (in /lib/libc-2.3.6.so)
==5957== 13,824 bytes in 3 blocks are definitely lost in loss record 71 of 85 ==5957== at 0x481CE9E: calloc (in /usr/lib/valgrind/x86-linux/vgpreload_memcheck.so) ==5957== by 0x485F215: mad_layer_III (in /usr/lib/libmad.so.0.2.1) ==5957== by 0x485429A: mad_frame_decode (in /usr/lib/libmad.so.0.2.1) ==5957== by 0x7775: decode_data (server.c:318) ==5957== by 0x9387: server (server.c:980) ==5957== by 0x4874164: pthread_start_thread (in /lib/libpthread-0.10.so) ==5957== by 0x4A176E9: clone (in /lib/libc-2.3.6.so)
One would assume this is just caused by memory not being free()'d up when the program exits, but the longer the server has run, the more memory was definitely lost, indicating that the memory loss originates here.
I use the mad library in such a way that I manually line up data and decode this data per frame, which is required to ensure certain functionality of the streaming server. I made sure that when I reinitialize the decoder every time I run the mad_*_finish() operations on all the structs I use before running mad_*_init() again. However, I suspect that not all allocated memory is freed as it should.
Am I missing something or could it be that using mad this way hasn't been tested excessively enough and is it true that the memory alllocated in mad_layer_III() on lines 2530 and 2538 (libmad version 0.15.1b) is not freed properly? I would be delighted to either get confirmation on this or some advice on how to solve this issue. If more information is required I will do my best to provide this.
Regards,
Niels Roosen
On Sep 26, 2007, at 3:40 AM, Niels Roosen wrote:
Am I missing something or could it be that using mad this way hasn't been tested excessively enough and is it true that the memory alllocated in mad_layer_III() on lines 2530 and 2538 (libmad version 0.15.1b) is not freed properly? I would be delighted to either get confirmation on this or some advice on how to solve this issue.
The memory allocated in mad_layer_III() is freed in mad_frame_finish () and mad_stream_finish(); you should make sure these are definitely being called on the same structures you pass to mad_frame_decode().
Since you say you are making these calls, I'm at a loss to offer any specific advice without seeing your code.
Rob Leslie wrote:
On Sep 26, 2007, at 3:40 AM, Niels Roosen wrote:
Am I missing something or could it be that using mad this way hasn't been tested excessively enough and is it true that the memory alllocated in mad_layer_III() on lines 2530 and 2538 (libmad version 0.15.1b) is not freed properly? I would be delighted to either get confirmation on this or some advice on how to solve this issue.
The memory allocated in mad_layer_III() is freed in mad_frame_finish() and mad_stream_finish(); you should make sure these are definitely being called on the same structures you pass to mad_frame_decode().
Since you say you are making these calls, I'm at a loss to offer any specific advice without seeing your code.
What I do is initialize the decoder, decode incoming data per frame until the client disconnects, and then, when another client connects, run the mad_stream_finish() and mad_frame_finish() on the pointers to the relevant structures before running the initialization functions again, in order to clean up any remaining data in the decoder. I took this from some example code but unfortunately don't know what any more, but I suppose that is how it should be done.
I will post the relevant code tomorrow, should I do that as attachment or inline?
Regards,
Niels
Rob Leslie wrote:
On Sep 26, 2007, at 2:55 PM, Niels Roosen wrote:
I will post the relevant code tomorrow, should I do that as attachment or inline?
Either is fine unless it's long, in which case attachment is probably preferred.
You're not going to believe this, but I seem to have solved the problem. While scrutinizing the relevant code for posting I discovered a rather odd construct in my code (don't know why I didn't see that before) and after some more investigation I discovered that I was lying earlier. In the initialization function (which is also responsible for calling the mad_*_finish() functions if necessary) the condition for actually calling the cleanup functions was not working at all! This was because I mixed up something during development and clearly never tested this properly. So I fixed that and didn't get -any- references to lost memory in libmad anymore \o/.
Anyway, to summarize: yet another case of tunnel vision, so you try to explain your problem to someone else and during that process find out what is going wrong. So thanks nevertheless, I guess I needed this :-)
So I humbly apologize for being such a retard. Nevertheless, mad rocks my socks, it is fast and reliable unlike the mpg123 decoder which often fails to synchronize frames, miserably failing to decode an entire stream... :-)
Cheers,
Niels
-
Niels Roosen -
Rob Leslie