I found libxml2 error (memory leak in multithreaded parsing)?

Question

I am really working on data processing code using libxml2 . I am stuck on a memory leak that cannot be removed. Here is the minimal code to create it:

#include <stdlib.h> #include <stdio.h> #include <libxml/parser.h> #include <libxml/tree.h> #include <omp.h> int main(void) { xmlDoc *doc; int tn; char fname[32]; omp_set_num_threads(2); xmlInitParser(); #pragma omp parallel private(doc,tn,fname) { tn = omp_get_thread_num(); sprintf(fname,"testdoc%d.xml",tn); doc = xmlReadFile(fname,NULL,0); printf("document %s parsed on thread %d (%p)\n",fname,tn,doc); xmlFreeDoc(doc); } xmlCleanupParser(); return EXIT_SUCCESS; } 

At run time, the output is:

 document testdoc0.xml parsed on thread 0 (0x1005413a0) document testdoc1.xml parsed on thread 1 (0x1005543c0) 

confirming that we really have multithreading and that the doc really private in a parallel area. You may notice that I correctly applied thread safety instructions to use libxml2 ( http://xmlsoft.org/threads.html ). Valgrind reports:

 HEAP SUMMARY: in use at exit: 9,000 bytes in 8 blocks total heap usage: 956 allocs, 948 frees, 184,464 bytes allocated 968 bytes in 1 blocks are definitely lost in loss record 6 of 8 at 0x1000107AF: malloc (vg_replace_malloc.c:236) by 0x1000B2590: xmlGetGlobalState (in /opt/local/lib/libxml2.2.dylib) by 0x1000B1A18: __xmlDefaultSAXHandler (in /opt/local/lib/libxml2.2.dylib) by 0x100106D18: xmlDefaultSAXHandlerInit (in /opt/local/lib/libxml2.2.dylib) by 0x100041BE7: xmlInitParserCtxt (in /opt/local/lib/libxml2.2.dylib) by 0x100042145: xmlNewParserCtxt (in /opt/local/lib/libxml2.2.dylib) by 0x10004615E: xmlCreateURLParserCtxt (in /opt/local/lib/libxml2.2.dylib) by 0x10005B56B: xmlReadFile (in /opt/local/lib/libxml2.2.dylib) by 0x100000E03: main.omp_fn.0 (in ./xtest) by 0x100028FA3: gomp_thread_start (in /opt/local/lib/gcc44/libgomp.1.dylib) by 0x1001E8535: _pthread_start (in /usr/lib/libSystem.B.dylib) by 0x1001E83E8: thread_start (in /usr/lib/libSystem.B.dylib) LEAK SUMMARY: definitely lost: 968 bytes in 1 blocks indirectly lost: 0 bytes in 0 blocks possibly lost: 0 bytes in 0 blocks still reachable: 8,032 bytes in 7 blocks suppressed: 0 bytes in 0 blocks Reachable blocks (those to which a pointer was found) are not shown. To see them, rerun with: --leak-check=full --show-reachable=yes 

This works for me regardless of the XML document used. I am using libxml 2.7.8 on Mac OS X 10.6.5 with gcc 4.4.5.

Can anyone reproduce this error?

Thanks,

Antonin

Answer 1

You should probably include this on the libxml2 mailing list.

http://mail.gnome.org/mailman/listinfo/xml

Answer 2

On the website above (http://xmlsoft.org/threads.html):

Starting with 2.4.7, libxml2 allows parallel processing of parallel streams in parallel analysis of different documents.

Your example apparently uses xmlReadFile for one document (testdoc.xml) for each stream. The following is said:

Please note that thread security cannot be ensured for multiple threads sharing the same document, blocking must be done at the application level ...