I start with pthreads in C, and I am also a maniac who, as I can, can write my code as "error free".
Despite trying to be more careful, valgrind tells me that I am a memory leak, regardless of the weather:
- I create linked threads that I join the completion (code snippet 1)
- I create linked threads that I disconnect after creating (code snippet 2)
- I create separate threads (code snippet 3)
I know this has already been discussed (see this , this , and also this ), but I'm still wondering:
- Why am I not getting errors on some launches?
- Why is there a random amount of common mallocs () when working with individual threads? & L; <response provided by nos, code snippet "fixed" with added delay in main ()
- Why does a “memory leak” persist even when working with individual threads? & L; <same as 2.
As I understand from the previous answers and the valgrind trail, pthread_create () is the main reason, expanding the stack used by threads, as needed, and reusing it from time to time, thus several missing frees. But what is less clear is why it depends on execution and why it happens when creating separate threads. As I saw from certain answers, comments, as well as from a person, resources from a dedicated stream will be released after the stream ends. I tried various settings to get around this (added sleep time to the end of each thread, to the end of the main thread, increased stack size, added more “work” ...), but this did not change the end result a lot. Also, why is there a random number of common "mallocs ()" when working with individual threads, will valgrind not lose some of the individual threads? It also does not depend on the size of the stack.
The code provided is an example example of the manager / workers model, for which the joinable / join () approach for thread management seems more appropriate imho.
Thanks for any education you could provide! I also hope that these (over-commented) snippets of code will be useful to anyone who wants to get started with pthreads.
- swappy
PS Sys info: gcc on a 64-bit debian arch
Code snippet 1 (merged threads are connected):
/* Running this multiple times with valgrind, I sometimes end with : - no errors (proper malloc/free balance) - 4 extra malloc vs free (most frequently) The number of mallocs() is more conservative and depends on the number of threads. */ #include <stdlib.h> /* EXIT_FAILURE, EXIT_SUCCESS macros & the likes */ #include <stdio.h> /* printf() & the likes */ #include <pthread.h> /* test subject */ #define MAX_THREADS 100 /* Number of threads */ pthread_attr_t tattr; /* Thread attribute */ pthread_t workers[MAX_THREADS]; /* All the threads spawned by the main() thread */ /* A mock container structure to pass arguments around */ struct args_for_job_t { int tid; int status; }; /* The job each worker will perform upon creation */ void *job(void *arg) { /* Cast arguments in a proper container */ struct args_for_job_t *container; container = (struct args_for_job_t *)arg; /* A mock job */ printf("[TID - %d]\n", container->tid); /* Properly exit with status code tid */ pthread_exit((void *)(&container->status)); } int main () { int return_code; /* Will hold return codes */ void *return_status; /* Will hold return status */ int tid; /* Thread id */ struct args_for_job_t args[MAX_THREADS]; /* For thread safeness */ /* Initialize and set thread joinable attribute */ pthread_attr_init(&tattr); pthread_attr_setdetachstate(&tattr, PTHREAD_CREATE_JOINABLE); /* Spawn detached threads */ for (tid = 0; tid < MAX_THREADS; tid++) { args[tid].tid = tid; args[tid].status = tid; return_code = pthread_create(&workers[tid], &tattr, job, (void *)(&args[tid])); if (return_code != 0) { printf("[ERROR] Thread creation failed\n"); return EXIT_FAILURE; } } /* Free thread attribute */ pthread_attr_destroy(&tattr); /* Properly join() all workers before completion */ for(tid = 0; tid < MAX_THREADS; tid++) { return_code = pthread_join(workers[tid], &return_status); if (return_code != 0) { printf("[ERROR] Return code from pthread_join() is %d\n", return_code); return EXIT_FAILURE; } printf("Thread %d joined with return status %d\n", tid, *(int *)return_status); } return EXIT_SUCCESS; }
Code snippet 2 (separate threads after creation):
/* Running this multiple times with valgrind, I sometimes end with : - no errors (proper malloc/free balance) - 1 extra malloc vs free (most frequently) Most surprisingly, it seems there is a random amount of overall mallocs */ #include <stdlib.h> /* EXIT_FAILURE, EXIT_SUCCESS macros & the likes */ #include <stdio.h> /* printf() & the likes */ #include <pthread.h> /* test subject */ #include <unistd.h> #define MAX_THREADS 100 /* Number of threads */ pthread_attr_t tattr; /* Thread attribute */ pthread_t workers[MAX_THREADS]; /* All the threads spawned by the main() thread */ /* A mock container structure to pass arguments around */ struct args_for_job_t { int tid; }; /* The job each worker will perform upon creation */ void *job(void *arg) { /* Cast arguments in a proper container */ struct args_for_job_t *container; container = (struct args_for_job_t *)arg; /* A mock job */ printf("[TID - %d]\n", container->tid); /* For the sake of returning something, not necessary */ return NULL; } int main () { int return_code; /* Will hold return codes */ int tid; /* Thread id */ struct args_for_job_t args[MAX_THREADS]; /* For thread safeness */ /* Initialize and set thread joinable attribute */ pthread_attr_init(&tattr); pthread_attr_setdetachstate(&tattr, PTHREAD_CREATE_JOINABLE); /* Spawn detached threads */ for (tid = 0; tid < MAX_THREADS; tid++) { args[tid].tid = tid; return_code = pthread_create(&workers[tid], &tattr, job, (void *)(&args[tid])); if (return_code != 0) { printf("[ERROR] Thread creation failed\n"); return EXIT_FAILURE; } /* Detach worker after creation */ pthread_detach(workers[tid]); } /* Free thread attribute */ pthread_attr_destroy(&tattr); /* Delay main() completion until all detached threads finish their jobs. */ usleep(100000); return EXIT_SUCCESS; }
Code snippet 3 (separate threads when creating):
/* Running this multiple times with valgrind, I sometimes end with : - no errors (proper malloc/free balance) - 1 extra malloc vs free (most frequently) Most surprisingly, it seems there is a random amount of overall mallocs */ #include <stdlib.h> /* EXIT_FAILURE, EXIT_SUCCESS macros & the likes */ #include <stdio.h> /* printf() & the likes */ #include <pthread.h> /* test subject */ #define MAX_THREADS 100 /* Number of threads */ pthread_attr_t tattr; /* Thread attribute */ pthread_t workers[MAX_THREADS]; /* All the threads spawned by the main() thread */ /* A mock container structure to pass arguments around */ struct args_for_job_t { int tid; }; /* The job each worker will perform upon creation */ void *job(void *arg) { /* Cast arguments in a proper container */ struct args_for_job_t *container; container = (struct args_for_job_t *)arg; /* A mock job */ printf("[TID - %d]\n", container->tid); /* For the sake of returning something, not necessary */ return NULL; } int main () { int return_code; /* Will hold return codes */ int tid; /* Thread id */ struct args_for_job_t args[MAX_THREADS]; /* For thread safeness */ /* Initialize and set thread detached attribute */ pthread_attr_init(&tattr); pthread_attr_setdetachstate(&tattr, PTHREAD_CREATE_DETACHED); /* Spawn detached threads */ for (tid = 0; tid < MAX_THREADS; tid++) { args[tid].tid = tid; return_code = pthread_create(&workers[tid], &tattr, job, (void *)(&args[tid])); if (return_code != 0) { printf("[ERROR] Thread creation failed\n"); return EXIT_FAILURE; } } /* Free thread attribute */ pthread_attr_destroy(&tattr); /* Delay main() completion until all detached threads finish their jobs. */ usleep(100000); return EXIT_SUCCESS; }
Valgrind output for code snippet 1 (merged threads and memory leak)
==27802== ==27802== HEAP SUMMARY: ==27802== in use at exit: 1,558 bytes in 4 blocks ==27802== total heap usage: 105 allocs, 101 frees, 28,814 bytes allocated ==27802== ==27802== Searching for pointers to 4 not-freed blocks ==27802== Checked 104,360 bytes ==27802== ==27802== 36 bytes in 1 blocks are still reachable in loss record 1 of 4 ==27802== at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==27802== by 0x400894D: _dl_map_object (dl-load.c:162) ==27802== by 0x401384A: dl_open_worker (dl-open.c:225) ==27802== by 0x400F175: _dl_catch_error (dl-error.c:178) ==27802== by 0x4013319: _dl_open (dl-open.c:639) ==27802== by 0x517F601: do_dlopen (dl-libc.c:89) ==27802== by 0x400F175: _dl_catch_error (dl-error.c:178) ==27802== by 0x517F6C3: __libc_dlopen_mode (dl-libc.c:48) ==27802== by 0x4E423BB: pthread_cancel_init (unwind-forcedunwind.c:53) ==27802== by 0x4E4257B: _Unwind_ForcedUnwind (unwind-forcedunwind.c:130) ==27802== by 0x4E4069F: __pthread_unwind (unwind.c:130) ==27802== by 0x4E3AFF4: pthread_exit (pthreadP.h:265) ==27802== ==27802== 36 bytes in 1 blocks are still reachable in loss record 2 of 4 ==27802== at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==27802== by 0x400B7EC: _dl_new_object (dl-object.c:161) ==27802== by 0x4006805: _dl_map_object_from_fd (dl-load.c:1051) ==27802== by 0x4008699: _dl_map_object (dl-load.c:2568) ==27802== by 0x401384A: dl_open_worker (dl-open.c:225) ==27802== by 0x400F175: _dl_catch_error (dl-error.c:178) ==27802== by 0x4013319: _dl_open (dl-open.c:639) ==27802== by 0x517F601: do_dlopen (dl-libc.c:89) ==27802== by 0x400F175: _dl_catch_error (dl-error.c:178) ==27802== by 0x517F6C3: __libc_dlopen_mode (dl-libc.c:48) ==27802== by 0x4E423BB: pthread_cancel_init (unwind-forcedunwind.c:53) ==27802== by 0x4E4257B: _Unwind_ForcedUnwind (unwind-forcedunwind.c:130) ==27802== ==27802== 312 bytes in 1 blocks are still reachable in loss record 3 of 4 ==27802== at 0x4C29DB4: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==27802== by 0x4010B59: _dl_check_map_versions (dl-version.c:300) ==27802== by 0x4013E1F: dl_open_worker (dl-open.c:268) ==27802== by 0x400F175: _dl_catch_error (dl-error.c:178) ==27802== by 0x4013319: _dl_open (dl-open.c:639) ==27802== by 0x517F601: do_dlopen (dl-libc.c:89) ==27802== by 0x400F175: _dl_catch_error (dl-error.c:178) ==27802== by 0x517F6C3: __libc_dlopen_mode (dl-libc.c:48) ==27802== by 0x4E423BB: pthread_cancel_init (unwind-forcedunwind.c:53) ==27802== by 0x4E4257B: _Unwind_ForcedUnwind (unwind-forcedunwind.c:130) ==27802== by 0x4E4069F: __pthread_unwind (unwind.c:130) ==27802== by 0x4E3AFF4: pthread_exit (pthreadP.h:265) ==27802== ==27802== 1,174 bytes in 1 blocks are still reachable in loss record 4 of 4 ==27802== at 0x4C29DB4: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==27802== by 0x400B57D: _dl_new_object (dl-object.c:77) ==27802== by 0x4006805: _dl_map_object_from_fd (dl-load.c:1051) ==27802== by 0x4008699: _dl_map_object (dl-load.c:2568) ==27802== by 0x401384A: dl_open_worker (dl-open.c:225) ==27802== by 0x400F175: _dl_catch_error (dl-error.c:178) ==27802== by 0x4013319: _dl_open (dl-open.c:639) ==27802== by 0x517F601: do_dlopen (dl-libc.c:89) ==27802== by 0x400F175: _dl_catch_error (dl-error.c:178) ==27802== by 0x517F6C3: __libc_dlopen_mode (dl-libc.c:48) ==27802== by 0x4E423BB: pthread_cancel_init (unwind-forcedunwind.c:53) ==27802== by 0x4E4257B: _Unwind_ForcedUnwind (unwind-forcedunwind.c:130) ==27802== ==27802== LEAK SUMMARY: ==27802== definitely lost: 0 bytes in 0 blocks ==27802== indirectly lost: 0 bytes in 0 blocks ==27802== possibly lost: 0 bytes in 0 blocks ==27802== still reachable: 1,558 bytes in 4 blocks ==27802== suppressed: 0 bytes in 0 blocks ==27802== ==27802== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 2 from 2)
Valgrind output for code snippet 1 (no mem-leak, a few runs later)
--29170-- Discarding syms at 0x64168d0-0x6426198 in /lib/x86_64-linux-gnu/libgcc_s.so.1 due to munmap() ==29170== ==29170== HEAP SUMMARY: ==29170== in use at exit: 0 bytes in 0 blocks ==29170== total heap usage: 105 allocs, 105 frees, 28,814 bytes allocated ==29170== ==29170== All heap blocks were freed -- no leaks are possible ==29170== ==29170== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 2 from 2) --29170-- --29170-- used_suppression: 2 dl-hack3-cond-1 ==29170== ==29170== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 2 from 2)