Geek Answers Handbook

How to bypass memory leak in provider dll used in Python?

I am using the C provider API for a piece of business software, loading my library using the Python ctypes module.

After deploying the software that I wrote, I found that the provider library wasted memory in a consistent and predictable manner depending on the number of calls to a particular function that was part of their API.

I even duplicated a leak in a C program that does not use heap allocation.

I contacted the seller about this issue and they said that they are working on it, but I probably cannot really expect a fix until the next version of the software.

I had the idea to restart the provider dll after a certain threshold of calls to the leak function, but this did not release the missed memory.

I found that I could make the library unload like this:

_ctypes.FreeLibrary(vendor_dll._handle)

This frees up memory, but causes the interpreter to crash, seemingly randomly, after a few minutes of using the provider API.

I found this problem in Pugon Tracker which describes my situation: https://bugs.python.org/issue14597

It seems that if there is still an open link to the library, forcing its unloading will inevitably lead to the collapse of the Python interpreter.

In the worst case, I think I can load the provider library into a separate process, proxy requests using the multiprocessor queue and set a watchdog timer to recreate the process if the interpreter dies.

?

+4
1

, , Pyro4, :

class LibraryWorker(multiprocessing.Process):
    def __init__(self):
        super().__init__()

    def run(self):
        self.library = ctypes.windll.LoadLibrary(
            'vendor_library.dll')

        Pyro4.serveSimple(
            {self, 'library'},
            ns=False)

    def lib_func(self):
        res = self.library.func()
        return res

, ctypes , .

, . , , .

+1

More articles:


All Articles