Idiomatic Rust Plugin System

Question

I want to pass external code to a plugin system. Inside my project, I have a Provider tag, which is the code for my plugin system. If you activate the "consumer" function, you can use plugins; if you do not, you are the author of the plugins.

I want plugin authors to get their code into my program by compiling into a shared library. Is a shared library a good design decision? Plugin restriction uses Rust anyway.

Does the host plugin have to go through path C to load the shared library: loading an unlisted function?

I just want the authors to use the Provider tag to implement their plugins and this. Looking at sharedlib and libloading , it seems impossible to load plugins using the idiomatic Rust method.

I would just like to load feature objects in my ProviderLoader :

 // lib.rs pub struct Sample { ... } pub trait Provider { fn get_sample(&self) -> Sample; } pub struct ProviderLoader { plugins: Vec<Box<Provider>> } 

When the program is sent, the file tree will look like this:

 . ├── fancy_program.exe └── providers ├── fp_awesomedude.dll └── fp_niceplugin.dll 

Is it possible if plugins are compiled into shared libraries? This will also affect the decision of the plugin container type.

Do you have any other ideas? Maybe I'm wrong, so shared libraries are not the holy grail.

I first posted this on the Rust forum. A friend advised me to try a stack overflow.

Answer 1

There is no official plugin system, and you will not be able to make plugins loaded at run time in pure Rust. I saw some discussions about creating a native plug-in system, but so far nothing has been decided, and perhaps this will never happen. You can use one of the following solutions:

• You can extend your code with native dynamic libraries using FFI . To use C ABI, you must use repr(C) , the no_mangle attribute, extern , etc. You will find more information on finding Rust FFI on the Internet. With this solution, you must use raw pointers: they do not have a security guarantee (i.e. you must use unsafe code).

  Of course, you can write your own dynamic library in Rust, but in order to load it and call functions, you must go through C ABI. This means that Rust's security guarantees do not apply here. In addition, you cannot use the highest level Rust functions like trait , enum , etc. Between library and binary.

• You can use the scripting language intended for the Rust extension as gluon or dyon . You can dynamically add functions to your code and execute them with the same guarantees as in Rust. This, in my opinion, is an easier way: if you have a choice, and if the execution speed is not critical, use this to avoid complex C / Rust interfaces .

  This last point is true for every scripting language not even directly related to Rust, if they are supported by the Rust box, like Lua, Python , Javascript, etc.

Answer 2

Introduction

I myself sorted things in this direction, and I found there a little official documentation, so I decided to play!

First of all, let me note that, since in these properties of insignificant words, please do not rely on any code here if you are trying to confuse airplanes in the air or nuclear missiles, at least not without more comprehensive testing than me done. I am not responsible if the code here deletes your OS and e-mail with an erroneous tearful admission of the murder of the Zodiac of your local police; we are on the edge of Rust here, and everything can change from one release or toolchain to another.

You can view my experiments in the following Github repository: Platform for the Rust plugin. This code is not particularly reliable, but with minor changes to PLUGIN_DIR static in host/src/lib.rs you can load debugging / release plugins and switch between .so / .dylib / .dll on the OS. I personally tested this on the stable version of Rust 1.20 both in debugging settings and for release on Windows 10 ( stable-x86_64-pc-windows-msvc ) and Cent OS 7 ( stable-x86_64-unknown-linux-gnu ). To check, you will have to manually cargo build (--release) box plugin , and then the cargo test (--release) box host .

An approach

The approach I used was a common common box, which was explicitly designated as a dependency, defining the general definitions of struct and trait . At first I was going to check for the existence of a structure with the same structure or trait with the same definitions defined independently in both libraries, but I refused it because it is too fragile and you will not want to make it into a real design. However, if someone wants to check this out, feel free to do the PR in the repository above, and I will update this answer.

In addition, the Rust plugin was declared dylib . I am not sure how compiling how cdylib will interact, since I think it will mean that when loading the plugin there are two versions of the standard Rust library (where I assume that cdylib statically links Rust stdlib to a common object).

Test

General notes

• The structures I #repr(C) were not declared #repr(C) . This can provide an extra layer of security by guaranteeing the layout, but I was very curious to write “clean” Rust plugins with a minimal amount of “Rust like C processing”. We already know that you can use Rust through FFI, wrapping things in opaque pointers, hand drops, etc., Therefore, this is not very useful for checking this.
• The used function signature pub fn foo(args) -> output with the directive #[no_mangle] , it turned out that rustfmt automatically changes extern "Rust" fn to just fn . I'm not sure I agree with this in this case, since they are certainly “external” functions here, but I will choose to comply with rustfmt .
• Remember that although this is Rust, it has insecurity because libloading (or the unstable DynamicLib functionality) will not enter characters for you. At first, I thought my Vec test proved that you couldn’t pass Vecs between the host and the plugin until I realized that I had Vec<i32> , and on the other, Vec<usize>
• Interestingly, several times I pointed out the optimized test build to an unoptimized plugin and vice versa, and it still worked. Nevertheless, I still cannot honestly recommend creating plugins and host applications with various instrumental goals, and even if I cannot promise that for some reason rustc / llvm will not decide to make certain optimizations on one version of the structure, not the other. Also, I'm not sure if this means that type passing through FFI prevents certain optimizations from occurring, such as Null Pointer Optimization.
• You are still limited to calls to the bare function, not Foo::bar due to lack of name. In addition, due to the fact that functions with feature boundaries are monomorphic, generic functions and structures are also absent. The compiler cannot know what you are going to call foo<i32> , so no foo<i32> will be generated. Any functions above the plugin border should only accept specific types and return only specific types.
• Likewise, you should be careful with lifetime for the same reasons, since there is no static verification of lifespan. Rust is forced to believe you when you say that a function returns &'a when it really is &'b .

Native rust

The first tests that I performed were not in arbitrary structures; just clean, native types of rust. This will give a basic level, if possible. I chose three basic types: &mut i32 , &mut Vec and Option<i32> -> Option<i32> . All of them were selected for very specific reasons: &mut i32 , because it tests the link, &mut Vec , because it checks the heap growth from the memory allocated in the main application, and Option as a double testing goal, passing by moving and matching a simple enumeration.

All three work as expected. Link mutation mutates the value, clicking on Vec works correctly, and Option works correctly, Some or None .

Definition of the overall structure

This meant checking if you could pass an unstructured structure with a common definition on both sides between the plugin and the host. This works as expected, but, as mentioned in the General Notes section, cannot promise that Rust will not be able to optimize and / or optimize the structure definition on the one hand, and not the other. Always check your specific use case and use CI if it changes.

Object with nested objects

This test uses a structure whose definition is determined only on the side of the plugin , but implements the attribute defined in the general box and returns Box<Trait> . This works as expected. The call to trait_obj.fun() works correctly.

At first, I assumed that there would be problems with dropping, without making the dash explicitly have Drop as a binding, but it turned out that Drop was also called correctly (this was confirmed by setting the value of the variable declared on the test stack through the raw pointer from the struct Drop function). (Naturally, I know that Drop always called even with feature objects in Rust, but I was not sure that dynamic libraries would complicate it).

Note

I have not tested what happens if you download the plugin, create an attribute object, and then release the plugin (which will probably close it). I can only assume that this is potentially disastrous. I recommend keeping the plugin open until the object's object is saved.

Notes

Plugins work exactly as you expect, just tying the box naturally, albeit with some limitations and traps. While you are testing, I think this is a very natural way. This makes loading characters more bearable, for example, if you need to load the new function and then get an object object that implements the interface. It also avoids unpleasant C memory leaks because you could not or did not forget to load the Drop / free function. However, be careful and always check!