Call User Agreement for P / Invoke and C #

Question

I have a business case in which I need to specify my own calling convention when using P / Invoke. In particular, I have an outdated dll that uses a non-standard ABI, and I need to specify a calling convention for each function.

For example, one function in this DLL takes its first two arguments through EAX and EBX, and the rest through the stack. Another function takes one argument through ECX, and the rest is on the stack. I have several hundred such functions, and I would like to avoid writing my own intermediate DLL module to access these functions.

My other option would be to manually use my own custom P / Invoke, which is undesirable for obvious reasons.

Any help is appreciated, thanks

Answer 1

I don’t understand what you mean with custom P / Invoke, but I don’t see how you could leave without uncontrolled C ++ with built-in assembly. However, since almost everything is passed as 32-bit values, you can get away with writing only one proxy for each function signature, as well as for each function. Or you can write a code generator that generates proxies from XML. I do not see this version being too unwanted, since all the functions of the proxy server will be very simple:

int RealFunction(int param1, const char * param2, char param 3);

int MyFunction(int param1, int param2, int param3) { // argument types do not matter as long as they are not doubles or structures
   __asm {
      mov eax, param1
      mov ebx, param2
      push param3
      call RealFunction
      ; depending on calling convention, you might need to do add esp, 12 here
      ; if RealFunction does not return its result in eax, you will need to do mov eax, <wherever the return value is> here
   }
}

Answer 2

, , , DLL. , , .

Answer 3

I learned about calling conventions some time ago and wrote code to convert calls. The code is called from a special C # shell, the wrapper library uses emit reflection (cannot make Marshal.getdelegateforfunctionpointer work) to emit a new p / invoke method for the special naked stub method. It captures the parameters and then calls the actual method.

Here is the c code. I don't have a C # part :( I was learning assembler too at that time, so the code could suck :)

typedef struct
{
    USHORT ParameterOneOffset;  // The offset of the first parameter in dwords starting at one
    USHORT ParameterTwoOffset;  // The offset of the second parmaeter in dwords starting at one
} FastCallParameterInfo;



    __declspec( naked,dllexport ) void __stdcall InvokeFast()
{
    FastCallParameterInfo paramInfo;
    int functionAddress;
    int retAddress;
    int paramOne, paramTwo;
    __asm
    {
        // Pop the return address and parameter info.  Store in memory.
        pop retAddress;
        pop paramInfo;
        pop functionAddress;

        // Check if any parameters should be stored in edx                          
        movzx ecx, paramInfo.ParameterOneOffset;     
        cmp ecx,0;
        je NoRegister;  

        // Calculate the offset for parameter one.
        movzx ecx, paramInfo.ParameterOneOffset;    // Move the parameter one offset to ecx
        dec ecx;                                    // Decrement by 1
        mov eax, 4;                                 // Put 4 in eax
        mul ecx;                                    // Multiple offset by 4

        // Copy the value from the stack on to the register.
        mov ecx, esp;                               // Move the stack pointer to ecx
        add ecx, eax;                               // Subtract the offset.
        mov eax, ecx;                               // Store in eax for later.
        mov ecx, [ecx];                             // Derefernce the value
        mov paramOne, ecx;                          // Store the value in memory.

        // Fix up stack
        add esp,4;                                  // Decrement the stack pointer
        movzx edx, paramInfo.ParameterOneOffset;    // Move the parameter one offset to edx
        dec edx;                                    // Decrement by 1
        cmp edx,0;                                  // Compare offset with zero
        je ParamOneNoShift;                         // If first parameter then no shift.

    ParamOneShiftLoop:
        mov ecx, eax;
        sub ecx, 4;
        mov ecx, [ecx]
        mov [eax], ecx;                             // Copy value over
        sub eax, 4;                                 // Go to next 
        dec edx;                                    // decrement edx
        jnz ParamOneShiftLoop;                      // Loop
    ParamOneNoShift:
        // Check if any parameters should be stored in edx                          
        movzx ecx, paramInfo.ParameterTwoOffset;     
        cmp ecx,0;
        je NoRegister;  

        movzx ecx, paramInfo.ParameterTwoOffset;    // Move the parameter two offset to ecx
        sub ecx, 2;                                 // Increment the offset by two.  One extra for since we already shifted for ecx
        mov eax, 4;                                 // Put 4 in eax
        mul ecx;                                    // Multiple by 4

        // Copy the value from the stack on to the register.
        mov ecx, esp;                               // Move the stack pointer to ecx
        add ecx, eax;                               // Subtract the offset.
        mov eax, ecx;                               // Store in eax for later.
        mov ecx, [ecx];                             // Derefernce the value
        mov paramTwo, ecx;                          // Store the value in memory.           

        // Fix up stack
        add esp,4;                                  // Decrement the stack pointer
        movzx edx, paramInfo.ParameterTwoOffset;    // Move the parameter two offset to ecx
        dec edx;                                    // Decrement by 1
        cmp edx,0;                                  // Compare offset with zero
        je NoRegister;                              // If first parameter then no shift.
    ParamTwoShiftLoop:
        mov ecx, eax;
        sub ecx, 4;
        mov ecx, [ecx]
        mov [eax], ecx;                             // Copy value over
        sub eax, 4;                                 // Go to next 
        dec edx;                                    // decrement edx
        jnz ParamTwoShiftLoop;                      // Loop


    NoRegister:
        mov ecx, paramOne;                          // Copy value from memory to ecx register
        mov edx, paramTwo;                          // 
        push retAddress;
        jmp functionAddress;
    }
}