Why does COM CoInitializeSecurity fail in my DLL?

Question

I am currently learning VSHADOW.EXE 3.0 from the MS Windows SDK 6.1. I made a version that can be compiled into a DLL that exports only one written function, which expects the command line to be a string, tokenize it, and then call the old wmain . DLL is not a COM server.

It works exactly the same as the old one when it is compiled as an EXE, but does not work when it is compiled as a DLL because this call fails:

  CoInitializeSecurity(NULL, -1, NULL, NULL, RPC_C_AUTHN_LEVEL_PKT_PRIVACY, RPC_C_IMP_LEVEL_IDENTIFY, NULL, EOAC_NONE, NULL); 

which fails with HRESULT error 0x80010119 ( RPC_E_TOO_LATE , security must be initialized before any interfaces are marshalled or unmarshaled. It cannot be changed after initialization.)

I am running an exported function from a VB6 program, where a function is imported using Declare Function vss Lib vshadow.dll ...

Does the error mean that the VB6 program has already called CoInitializeSecurity ? What can I do against a mistake?

In addition, I have another question: why were the security values RPC_C_AUTHN_LEVEL_PKT_PRIVACY and RPC_C_IMP_LEVEL_IDENTIFY ? What effect will other settings have?

Answer 1

There are several standard COM calls that are not related to DLLs. Like CoInitializeEx (), a call that initializes COM for the stream. The DLL does not own the stream; it is powerless to override the state of the apartment that the EXE chose.

CoInitializeSecurity () is another, it is an exe task to call it. Only he knows what values ​​must pass, this is the one that defines the security policy. The DLL cannot, it knows nothing about the client process.