Dynamic code compilation

Question

I am working on a program that runs iterative fractal systems. I wanted to add functionality where someone could define their own iteration process and compile this code so that it works efficiently.

Currently, I do not know how to do this, and I would like to know what to read in order to learn how to do this.

The main program is written in C ++, and I am familiar with C ++. In fact, given most of the scenarios, I know how to convert it to assembly code that will achieve the goal, but I don't know how to take an extra step to convert it to machine code. If possible, I would like to dynamically compile the code, as I believe, many game system emulators work.

If it is unclear what I ask, tell me so that I can clarify.

Thanks!

Answer 1

Some CPU emulators process machine code as if it were byte code, and they compile JIT, almost as if it were Java. This is very effective, but it means that developers need to write a compiler version for each processor that the emulator is running on, and for each emulated processor.

This usually means that it only works on x86 and annoys anyone who would like to use something else.

They can also translate it into LLVM or Java byte code or .Net CIL, and then compile it, which will also work.

In your case, I'm not sure if this is the best way to go. I think I will do this using dynamic libraries. Create a directory that should contain "plugins" and let the user compile his own. Make your program a directory check and load each DLL or .so finds.

Doing this means that you spend less time compiling code compilers and more time actually getting the material.

Answer 2

Is it necessary to dynamically compose a subroutine, which should be in any particular language. If the answer to this question is “Yes, it should be C ++,” you probably were out of luck. C ++ is the worst choice for online recompilation.

Is the dynamic part of your application (fractal iterator procedure) the main processor bottleneck? If you can afford to use a language that is not compiled, you can probably save a ton of trouble. Lua and JavaScript are highly optimized interpreted languages ​​that work only several times slower than their own compiled code.

If you really need dynamic functionality that will be compiled for machine code, the best option would probably be to use clang / llvm. clang is a C / Objective-C interface developed by Apple (and several others) to do online, dynamic recompilation works well. llvm is a backend clan used to translate from portable bytecode to native machine code. Keep in mind that currently the clan does not support most of C ++, as this difficult language is suitable for the correct one.

Answer 3

If you can write your dynamic extensions in C (not C ++), you can find the Tiny C Compiler . It is available under LGPL, it is compatible with Windows and Linux, as well as a small executable file (or library) of ~ 100 kb in size for the preprocessor, compiler, linker and assembler, all of which it works very quickly. The disadvantage of this, of course, is that it cannot compare with the optimizations you can get with GCC. Another potential downside is that the X86 is only AFAIK.

If you decide to write an assembly, TCC can handle it - the documentation says it supports gas-like syntax and it supports X86 opcodes.

TCC also fully supports ANSI C and is almost fully compatible with C99.

In this case, you can either enable TCC as an executable file with your application, or use libtcc (there is not too much documentation for libtcc online, but it is available in the source package). In any case, you can use tcc to create dynamic or shared libraries or executables. If you went along the path of a dynamic library, you would simply add the Render function (or something else) and dlopen or LoadLibrary , and call Render to finally start the user-created rendering. Alternatively, you can make a standalone executable and popen and execute all your communication through standalone stdin and stdout .

Answer 4

Since you are generating pixels displayed on the screen, have you considered using HLSL with dynamic shader compilation? This will give you access to SIMD hardware designed for this kind of thing, as well as a dynamic compiler built right into DirectX.

Answer 5

LLVM should be able to do what you want to do. This allows you to generate a description of the program that you want to compile in an object-oriented manner, and then can compile this description of the program into native machine code at run time.

Answer 6

Nanojit is a pretty good example of what you want. It generates machine code from an intermediate langauge. This is C ++, and it is small and cross-platform. I did not use it very widely, but I liked to deal only with demos.

Answer 7

Select the code in a file and compile it as a dynamically loaded library, then load it and call it.

Answer 8

Is there a reason why you cannot use GPU solutions? It seems to scream for one.