Transition from std :: string, std :: ostream, etc. In the open library API

Question

For API / ABI compatibility in many toolchains with the same binary code, it is well known that STL, std::string containers and other standard library classes, such as iostreams, are verboten in public headers. (An exception to this is if you distribute one assembly for each version of the supported toolchains, one provides source code without binaries for compiling end users that are not preferred parameters in this case, or one translates to another container internally, so a different implementation std does not get into the library.)

If you already had a published library API that did not comply with this rule (request to a friend), what is the best way forward, while preserving as much compatibility as possible with previous versions, since I reasonably can and prefer compromise breakdowns, t? I need to support Windows and Linux.

Go to the level of compatibility with the ABI I'm looking for: I don't need to be insanely reliable in the future. Basically, I want to make only one binary library file for several popular Linux distributions for release. (I currently release one for each compiler, and sometimes special versions for a special distribution (RHEL vs Debian). The same applies to MSVC versions - one of the DLLs for all supported versions of MSVC would be ideal.) Secondly, t break the API in the bugfix release, I would like it to be compatible with ABI and replace DLL / SO replace without restoring the client application.

I have three cases with some preliminary sentences modeled after Qt to the extent.

Old public API:

 // Case 1: Non-virtual functions with containers void Foo( const char* ); void Foo( const std::string& ); // Case 2: Virtual functions class Bar { public: virtual ~Bar() = default; virtual void VirtFn( const std::string& ); }; // Case 3: Serialization std::ostream& operator << ( std::ostream& os, const Bar& bar ); 

Case 1: Non-virtual functions with containers

In theory, we can convert std::string to a class very similar to std::string_view , but under our library API / ABI control. It is converted to our library header from std::string , so the compiled library still accepts, but does not depend on the implementation of std::string and is backward compatible:

New API:

 class MyStringView { public: MyStringView( const std::string& ) // Implicit and inline { // Convert, possibly copying } MyStringView( const char* ); // Implicit // ... }; void Foo( MyStringView ); // Ok! Mostly backwards compatible 

Most client codes that don’t do anything abnormal, like taking a Foo address, will work without change. Similarly, we can create our own replacement std::vector , although in some cases it may incur a copy penalty.

Abseil ToW # 1 recommends starting with the utility code and working instead of launching it in the API. Any other tips or pitfalls here?

Case 2: Virtual Functions

But what about virtual functions? We break backward compatibility if we change the signature. I suppose we could leave the old one in place using final to force the break:

 // Introduce base class for functions that need to be final class BarBase { public: virtual ~BarBase() = default; virtual void VirtFn( const std::string& ) = 0; }; class Bar : public BarBase { public: void VirtFn( const std::string& str ) final { VirtFn( MyStringView( str ) ); } // Add new overload, also virtual virtual void VirtFn( MyStringView ); }; 

Now redefinition of the old virtual function will be interrupted at compile time, but calls to std::string will be automatically converted. Overrides should use the new version and break at compile time.

Any clues or pitfalls here?

Case 3: Serialization

I'm not sure what to do with iostreams. One option, at the risk of some inefficiency, is to define them in a line and redirect them through the lines:

 MyString ToString( const Bar& ); // I control this, could be a virtual function in Bar if needed // Here I publicly interact with a std object, so it must be inline in the header inline std::ostream& operator << ( std::ostream& os, const Bar& bar ) { return os << ToString( bar ); } 

If I made ToString() virtual function, then I can iterate over all Bar objects and call user overrides, since it depends only on MyString objects, which are defined in the header, where they interact with std objects, such as a stream.

Thoughts, pitfalls?

Answer 1

Level 1

Use a good string representation.

Do not use virtual overload std::string const& ; there is no reason for this. You are breaking ABI anyway. After recompiling them, they will see a new overload based on the string representation, unless they take and store pointers to virtual functions.

For a stream without going to the intermediate line, use the continue traversal style:

 void CPS_to_string( Bar const& bar, MyFunctionView< void( MyStringView ) > cps ); 

where cps is called repeatedly with partial buffers until the object serializes it. Write << on top of this (built in headers). There is some unavoidable overhead from a function pointer pointer.

Now use only virtual interfaces and never overload virtual methods and always add new methods to the end of vtable. Therefore, do not expose complex hierarchies. The vtable extension is a secure ABI; adding to the middle is not.

FunctionView is a simple clonal cloning of the std function, which is not the owner, whose state is void* and R(*)(void*,args&&...) , which must be stable ABI to pass through the library boundary.

 template<class Sig> struct FunctionView; template<class R, class...Args> struct FunctionView<R(Args...)> { FunctionView()=default; FunctionView(FunctionView const&)=default; FunctionView& operator=(FunctionView const&)=default; template<class F, std::enable_if_t<!std::is_same< std::decay_t<F>, FunctionView >{}, bool> = true, std::enable_if_t<std::is_convertible< std::result_of_t<F&(Args&&...)>, R>, bool> = true > FunctionView( F&& f ): ptr( std::addressof(f) ), f( [](void* ptr, Args&&...args)->R { return (*static_cast< std::remove_reference_t<F>* >(ptr))(std::forward<Args>(args)...); } ) {} private: void* ptr = 0; R(*f)(void*, Args&&...args) = 0; }; template<class...Args> struct FunctionView<void(Args...)> { FunctionView()=default; FunctionView(FunctionView const&)=default; FunctionView& operator=(FunctionView const&)=default; template<class F, std::enable_if_t<!std::is_same< std::decay_t<F>, FunctionView >{}, bool> = true > FunctionView( F&& f ): ptr( std::addressof(f) ), f( [](void* ptr, Args&&...args)->void { (*static_cast< std::remove_reference_t<F>* >(ptr))(std::forward<Args>(args)...); } ) {} private: void* ptr = 0; void(*f)(void*, Args&&...args) = 0; }; 

this allows common callbacks to pass through the API barrier.

 // f can be called more than once, be prepared: void ToString_CPS( Bar const& bar, FunctionView< void(MyStringView) > f ); inline std::ostream& operator<<( std::ostream& os, const Bar& bar ) { ToString_CPS( bar, [&](MyStringView str) { return os << str; }); return os; } 

and implement ostream& << MyStringView const& in the headers.

---

Level 2

Forward each operation from the C ++ API to the headers on the extern "C" pure-C function (that is, pass a StringView as a char const* ptrs pair). Export only extern "C" character set. Now changes to symbolic manipulations no longer interrupt ypur ABI.

C ABI is more stable than C ++, and by forcing you to break library calls into "C" call calls, you can make ABI changes obvious. Use the C ++ header glue to make things clean, C to make ABI solid.

You can keep your clean virtual interfaces if you are willing to take risks; use the same rules as above (simple hierarchies, without overloads, just add them to the end) and you will get decent ABI stability.

Answer 2

Containers

To accept strings and arrays as arguments to the function, use std::string_view and gsl::span respectively, or your own equivalents with stable ABI. Non-contiguos containers can be passed as any_iterator ranges.

To return by reference, you can use these classes again.

To return a sting value by value, you can return std::string_view to the global global stream object, which is valid until the next API call (for example, the std::ctime ). If necessary, the user must make a deep copy.

You can use the callback based API to return the container by value. Your API will call a user callback for each element of the returned container.

std::string_view , gsl::span and any_iterator or their equivalents should be implemented in the header files that are supplied with your library for your users.

Virtual functions

You can use the Pimpl idiom instead of classes with virtual functions in your library's API.

Serialization

It can be implemented as functions in header files that use the public API of your library and serialize / deserialize using IOStreams.