Return std :: string / std :: list from dll

Question

Short question.

I just got a dll that I have to interact with. Dll uses crt from msvcr90D.dll (notification D) and returns std :: strings, std :: lists and boost :: shared_ptr. The new / delete operator is not overloaded anywhere.

I assume that crt mixup (msvcr90.dll in the release build, or if one of the components is rebuilt with the new crt, etc.) will inevitably cause problems in the end, and the dll should be rewritten to avoid returning anything that could cause new / delete (i.e. everything that could cause delete in my code in the memory block that was allocated (possibly with another crt) in the dll).

Am I right or not?

Answer 1

The main thing to keep in mind is that the DLL contains code, not memory. The allocated memory belongs to the process (1). When you instantiate an object in your process, you call the constructor code. During this lifetime of the object, you will refer to other parts of the code (methods) to work with this memory of the object. Then, when the object leaves, the destructor code is called.

STL templates are not explicitly exported from dll. The code is statically linked to each dll. Therefore, when std :: string s is created in a.dll and passed to b.dll, each dll will have two different instances of the string :: copy method. the copy called in a.dll calls the copy method a.dll ... If we work with s in b.dll and call the copy, the copy method in b.dll will be called.

That's why in Simon's answer, he says:

Bad things will happen if you cannot always guarantee that the entire set of binaries are all built with the same toolbox.

because if for some reason a copy of string s differs between a.dll and b.dll, strange things will happen. Even worse, if the line itself is different between a.dll and b.dll, and the destructor in one knows to clear additional memory, which the other ignores ... it may be difficult for you to track memory leaks. Perhaps even worse ... a.dll may have been created against a completely different version of STL (i.e. STLPort), while b.dll is created using Microsoft STL.

So what should you do? Where we work, we strictly control the tools and set the parameters for each DLL. Therefore, when we develop internal dlls, we freely transfer STL templates. We still have problems that arise on a rare occasion, because someone misconfigured their project. However, we believe that the convenience of STL is worth the random problem that arises.

To expose dlls to third parties, this other story is complete. If you do not want to strictly require specific build settings from clients, you do not need to export STL templates. I do not recommend strictly applying your clients to specific build settings ... they may have another third-party tool that expects you to use completely opposite build settings.

(1) Yes, I know that static and locals are created or deleted when loading / unloading dlls.

Answer 2

I have this exact problem in the project I'm working on - the STL classes are passed to and from the DLLs. The problem is not only in different memory, but also in the fact that STL classes do not have a binary standard (ABI). For example, in debug builds, some STL implementations add additional debugging information to STL classes, such as sizeof(std::vector<T>) (release build)! = sizeof(std::vector<T>) (debug build). Oh! There is no hope that you can rely on the binary compatibility of these classes. Also, if your DLL was compiled in another compiler with some other STL implementation that used different algorithms, you may have a different binary format in versions.

The way I solved this problem is to use a template class called pod<T> (POD means Plain Old Data, such as chars and ints, which are usually passed between DLLs). The task of this class is to pack its template parameter into a consistent binary format, and then unpack it at the other end. For example, instead of the function in the DLL that returns std::vector<int> , you return pod<std::vector<int>> . It specializes in a template for pod<std::vector<T>> , which expands the memory buffer and copies elements. It also provides an operator std::vector<T>() , so the return value can be transparently saved back to std :: vector by building a new vector, copying the saved elements into it and returning it. Since it always uses the same binary format, it can be safely compiled to split binary files and save binary data. An alternate name for pod might be make_binary_compatible .

Here is the pod class definition:

 // All members are protected, because the class *must* be specialization // for each type template<typename T> class pod { protected: pod(); pod(const T& value); pod(const pod& copy); // no copy ctor in any pod pod& operator=(const pod& assign); T get() const; operator T() const; ~pod(); }; 

Here's the partial specialization for pod<vector<T>> - pay attention to the partial specialization, so this class works for any type T. Also note: it actually stores the pod<T> memory buffer, not just T - if the vector contained another type of STL, such as std :: string, we would like this to be binary compatible too!

 // Transmit vector as POD buffer template<typename T> class pod<std::vector<T> > { protected: pod(const pod<std::vector<T> >& copy); // no copy ctor // For storing vector as plain old data buffer typename std::vector<T>::size_type size; pod<T>* elements; void release() { if (elements) { // Destruct every element, in case contained other cr::pod<T>s pod<T>* ptr = elements; pod<T>* end = elements + size; for ( ; ptr != end; ++ptr) ptr->~pod<T>(); // Deallocate memory pod_free(elements); elements = NULL; } } void set_from(const std::vector<T>& value) { // Allocate buffer with room for pods of T size = value.size(); if (size > 0) { elements = reinterpret_cast<pod<T>*>(pod_malloc(sizeof(pod<T>) * size)); if (elements == NULL) throw std::bad_alloc("out of memory"); } else elements = NULL; // Placement new pods in to the buffer pod<T>* ptr = elements; pod<T>* end = elements + size; std::vector<T>::const_iterator iter = value.begin(); for ( ; ptr != end; ) new (ptr++) pod<T>(*iter++); } public: pod() : size(0), elements(NULL) {} // Construct from vector<T> pod(const std::vector<T>& value) { set_from(value); } pod<std::vector<T> >& operator=(const std::vector<T>& value) { release(); set_from(value); return *this; } std::vector<T> get() const { std::vector<T> result; result.reserve(size); // Copy out the pods, using their operator T() to call get() std::copy(elements, elements + size, std::back_inserter(result)); return result; } operator std::vector<T>() const { return get(); } ~pod() { release(); } }; 

Please note that the memory allocation functions used - pod_malloc and pod_free - are just malloc and free, but using the same function between all the DLLs. In my case, all the DLLs use malloc and are free from the EXE host, so they all use the same heap, which solves the heap memory problem. (Exactly how you understand this is up to you.)

Also note that you need specializations for pod<T*> , pod<const T*> and pod for all main types ( pod<int> , pod<short> , etc.) so that they can be saved in the "vector container" and the other block containers. They should be simple enough to write if you understand the example above.

This method means copying the entire object. However, you can pass references to pod types, since there is operator= , which is safe between binary files. However, there is no real transfer by reference, since the only way to change the type of an element is to copy it back to the original type, change it, and then repack it as a container. In addition, the created copies mean that this is not necessarily the fastest way, but it works.

However, you can also specialize your own types, which means you can effectively return complex types such as std::map<MyClass, std::vector<std::string>> , providing specialization for pod<MyClass> and partial specializations for std::map<K, V> , std::vector<T> and std::basic_string<T> (which you only need to write once).

Using the final result is as follows. The common interface is defined:

 class ICommonInterface { public: virtual pod<std::vector<std::string>> GetListOfStrings() const = 0; }; 

A DLL can implement this as such:

 pod<std::vector<std::string>> MyDllImplementation::GetListOfStrings() const { std::vector<std::string> ret; // ... // pod can construct itself from its template parameter // so this works without any mention of pod return ret; } 

And the caller, a separate binary, may call it the following:

 ICommonInterface* pCommonInterface = ... // pod has an operator T(), so this works again without any mention of pod std::vector<std::string> list_of_strings = pCommonInterface->GetListOfStrings(); 

So, once it is configured, you can use it almost as if the pod class were not there.

Answer 3

I’m not sure that “everything that could be called new / delete” can be controlled by carefully using the equivalents of common pointers with the corresponding distribution / delete functions.

However, in the general case, I would not let the templates pass through the boundaries of the DLL - the implementation of the template class ends on both sides of the interface, which means that you can use another implementation. Bad things will happen if you cannot always guarantee that your entire set of binary files will be built with the same toolchain.

When I need this functionality, I often use the virtual interface class across the border. You can then provide wrappers for std::string , list , etc., which will allow you to use them safely through the interface. Then you can control the distribution, etc., using your implementation, or using shared_ptr .

Having said all this, one thing that I use in my DLLs is shared_ptr , since it is not too useful for. I have not had any problems yet, but everything was built using the same toolchain. I'm waiting for this to bite me, so no doubt it will. See the previous question: Using shared_ptr in dll interfaces

Answer 4

For std::string you can return using c_str . In the case of more complex things, the option may be something like

 class ContainerValueProcessor { public: virtual void operator()(const trivial_type& value)=0; }; 

Then (if you want to use std :: list), you can use the interface

 class List { public: virtual void processItems(ContainerValueProcessor&& proc)=0; }; 

Note that List can now be implemented with any container.