OK, so I'm trying to make some overlays for some extra buttons in a Direct X game.
I found a C ++ sample that overlays pretty well here: http://www.gamedev.net/topic/359794-c-direct3d-hooking-sample/
So, I started to convert it to Delphi. With some logging, I see that it starts the hook on the correct process and correctly clicks Direct3DCreate9 ().
Next, TMyDirect3D9 was created successfully. But the process drops out of here.
My educated assumption (based on some debugging in Ollydbg) is that when I return MyDirect3D9 back to the original process using the connected Direct3DCreate9 () and try to call one of the functions of the class (interface), it fails.
The following is the code. If I can give any other information to help me inform.
Main DLL:
library LeagueUtilityBox; {$R *.res} {$DEFINE DEBUG} uses Windows, APIHijack in 'APIHijack.pas', Direct3D9 in '..\DirectX 9.0\Direct3D9.pas', uSharedMem in '..\Misc\uSharedMem.pas', MyDirect3D9 in 'MyDirect3D9.pas', MyDirect3DDevice9 in 'MyDirect3DDevice9.pas', {$IFDEF DEBUG} SysUtils, uLog in '..\Misc\uLog.pas', {$ENDIF} uMisc in 'uMisc.pas'; var SharedMem : TSharedMem; D3DHook: SDLLHook; hHook : DWORD; MyDirect3D9 : TMyDirect3D9; function GetTargetProcess: String; const KeyBase : DWORD = HKEY_CURRENT_USER; KeyLocation : String = 'Software\LeagueUtilityBox'; var RegKey : HKEY; TargetProcess : Array[0..511] Of Char; Count : DWORD; begin Result := ''; If RegOpenKeyEx(KeyBase, PChar(KeyLocation), 0, KEY_QUERY_VALUE, RegKey) = ERROR_SUCCESS Then begin Count := 512; If RegQueryValueEx(RegKey, nil, nil, nil, @TargetProcess[0], @Count) = ERROR_SUCCESS Then begin Result := String(TargetProcess); end; end; end; type TDirect3DCreate9 = function(SDKVersion: LongWord): Pointer; stdcall; function MyDirect3DCreate9(SDKVersion: LongWord): Pointer; stdcall; var OldFunc : TDirect3DCreate9; D3D : PIDirect3D9; begin {$IFDEF DEBUG} WriteToLog('C:\LeagueUtilityBox.log', 'MyDirect3DCreate9 called'); {$ENDIF} Result := nil; OldFunc := TDirect3DCreate9(D3DHook.Functions[0].OrigFn); D3D := OldFunc(SDKVersion); If D3D <> nil Then begin {$IFDEF DEBUG} WriteToLog('C:\LeagueUtilityBox.log', 'D3D created: 0x' + IntToHex(DWORD(Pointer(D3D)), 8)); {$ENDIF} New(MyDirect3D9); MyDirect3D9 := TMyDirect3D9.Create(D3D); {$IFDEF DEBUG} WriteToLog('C:\LeagueUtilityBox.log', 'MyDirect3D9 Created'); {$ENDIF} Result := @MyDirect3D9; end; end; procedure InitializeHook; var Process : String; I : Integer; begin SetLength(Process, 512); GetModuleFileName(GetModuleHandle(nil), PChar(Process), 512); For I := Length(Process) DownTo 1 Do begin If Process[I] = '\' Then Break; end; Process := Copy(Process, I + 1, Length(Process)); If CompareString(LOCALE_SYSTEM_DEFAULT, NORM_IGNORECASE, PChar(GetTargetProcess), -1, PChar(Process), -1) = 2 Then begin {$IFDEF DEBUG} WriteToLog('C:\LeagueUtilityBox.log', 'Found target: ' + GetTargetProcess); {$ENDIF} With D3DHook Do begin Name := 'D3D9.DLL'; UseDefault := False; DefaultFn := nil; SetLength(Functions, 1); Functions[0].Name := 'Direct3DCreate9'; Functions[0].HookFn := @MyDirect3DCreate9; Functions[0].OrigFn := nil; end; {$IFDEF DEBUG} WriteToLog('C:\LeagueUtilityBox.log', 'About to hook: ' + String(AnsiString(D3DHook.Name))); {$ENDIF} HookAPICalls(@D3DHook); {$IFDEF DEBUG} WriteToLog('C:\LeagueUtilityBox.log', 'Hook completed: ' + String(AnsiString(D3DHook.Name))); {$ENDIF} end; end; procedure InitializeDLL; begin SharedMem := TSharedMem.Create('LeagueUtilityBox', 1024); Try hHook := PDWORD(SharedMem.Buffer)^; {$IFDEF DEBUG} WriteToLog('C:\LeagueUtilityBox.log', 'Initializing DLL: ' + IntToStr(hHook)); {$ENDIF} Finally SharedMem.Free; end; end; procedure UninitializeDLL; begin UnhookWindowsHookEx(hHook); end; function WindowsHookCallback(nCode: Integer; WPARAM: Integer; LPARAM: Integer): LRESULT; stdcall; begin Result := CallNextHookEx(hHook, nCode, WPARAM, LPARAM); end; procedure EntryPoint(Reason: DWORD); begin Case Reason Of DLL_PROCESS_ATTACH: begin InitializeDLL; InitializeHook; end; DLL_PROCESS_DETACH: begin UninitializeDLL; end; end; end; exports WindowsHookCallback; begin DLLProc := @EntryPoint; EntryPoint(DLL_PROCESS_ATTACH); end.
Custom IDirect3D9:
unit MyDirect3D9; interface uses Direct3D9, Windows, uMisc, uLog; type PMyDirect3D9 = ^TMyDirect3D9; TMyDirect3D9 = class(TInterfacedObject, IDirect3D9) private fD3D: PIDirect3D9; public constructor Create(D3D: PIDirect3D9); function QueryInterface(riid: REFIID; ppvObj: PPointer): HRESULT; stdcall; function _AddRef: DWORD; stdcall; function _Release: DWORD; stdcall; function RegisterSoftwareDevice(pInitializeFunction: Pointer): HResult; stdcall; function GetAdapterCount: LongWord; stdcall; function GetAdapterIdentifier(Adapter: LongWord; Flags: DWord; out pIdentifier: TD3DAdapterIdentifier9): HResult; stdcall; function GetAdapterModeCount(Adapter: LongWord; Format: TD3DFormat): LongWord; stdcall; function EnumAdapterModes(Adapter: LongWord; Format: TD3DFormat; Mode: LongWord; out pMode: TD3DDisplayMode): HResult; stdcall; function GetAdapterDisplayMode(Adapter: LongWord; out pMode: TD3DDisplayMode): HResult; stdcall; function CheckDeviceType(Adapter: LongWord; CheckType: TD3DDevType; AdapterFormat, BackBufferFormat: TD3DFormat; Windowed: BOOL): HResult; stdcall; function CheckDeviceFormat(Adapter: LongWord; DeviceType: TD3DDevType; AdapterFormat: TD3DFormat; Usage: DWord; RType: TD3DResourceType; CheckFormat: TD3DFormat): HResult; stdcall; function CheckDeviceMultiSampleType(Adapter: LongWord; DeviceType: TD3DDevType; SurfaceFormat: TD3DFormat; Windowed: BOOL; MultiSampleType: TD3DMultiSampleType; pQualityLevels: PDWORD): HResult; stdcall; function CheckDepthStencilMatch(Adapter: LongWord; DeviceType: TD3DDevType; AdapterFormat, RenderTargetFormat, DepthStencilFormat: TD3DFormat): HResult; stdcall; function CheckDeviceFormatConversion(Adapter: LongWord; DeviceType: TD3DDevType; SourceFormat, TargetFormat: TD3DFormat): HResult; stdcall; function GetDeviceCaps(Adapter: LongWord; DeviceType: TD3DDevType; out pCaps: TD3DCaps9): HResult; stdcall; function GetAdapterMonitor(Adapter: LongWord): HMONITOR; stdcall; function CreateDevice(Adapter: LongWord; DeviceType: TD3DDevType; hFocusWindow: HWND; BehaviorFlags: DWord; pPresentationParameters: PD3DPresentParameters; out ppReturnedDeviceInterface: IDirect3DDevice9): HResult; stdcall; end; implementation uses MyDirect3DDevice9; constructor TMyDirect3D9.Create(D3D: PIDirect3D9); begin {$IFDEF DEBUG} WriteToLog('C:\LeagueUtilityBox.log', 'TMyDirect3D9.Create'); {$ENDIF} fD3D := D3D; end; function TMyDirect3D9.QueryInterface(riid: REFIID; ppvObj: PPointer): HRESULT; stdcall; begin {$IFDEF DEBUG} WriteToLog('C:\LeagueUtilityBox.log', 'TMyDirect3D9.QueryInterface'); {$ENDIF} Result := fD3D^.QueryInterface(riid, ppvObj); end; function TMyDirect3D9._AddRef: DWORD; stdcall; begin {$IFDEF DEBUG} WriteToLog('C:\LeagueUtilityBox.log', 'TMyDirect3D9._AddRef'); {$ENDIF} Result := fD3D^._AddRef; end; function TMyDirect3D9._Release: DWORD; stdcall; var count : DWORD; begin {$IFDEF DEBUG} WriteToLog('C:\LeagueUtilityBox.log', 'TMyDirect3D9._Release'); {$ENDIF} count := fD3D^._Release; If count = 0 Then begin Self.Free; end; Result := count; end; function TMyDirect3D9.RegisterSoftwareDevice(pInitializeFunction: Pointer): HResult; stdcall; begin {$IFDEF DEBUG} WriteToLog('C:\LeagueUtilityBox.log', 'TMyDirect3D9.RegisterSoftwareDevice'); {$ENDIF} Result := fD3D^.RegisterSoftwareDevice(pInitializeFunction); end; function TMyDirect3D9.GetAdapterCount: LongWord; stdcall; begin {$IFDEF DEBUG} WriteToLog('C:\LeagueUtilityBox.log', 'TMyDirect3D9.GetAdapterCount'); {$ENDIF} Result := fD3D^.GetAdapterCount; end; function TMyDirect3D9.GetAdapterIdentifier(Adapter: LongWord; Flags: DWord; out pIdentifier: TD3DAdapterIdentifier9): HResult; stdcall; begin {$IFDEF DEBUG} WriteToLog('C:\LeagueUtilityBox.log', 'TMyDirect3D9.GetAdapterIdentifier'); {$ENDIF} Result := fD3D^.GetAdapterIdentifier(Adapter, Flags, pIdentifier); end; function TMyDirect3D9.GetAdapterModeCount(Adapter: LongWord; Format: TD3DFormat): LongWord; stdcall; begin {$IFDEF DEBUG} WriteToLog('C:\LeagueUtilityBox.log', 'TMyDirect3D9.GetAdapterModeCount'); {$ENDIF} Result := fD3D^.GetAdapterModeCount(Adapter, Format); end; function TMyDirect3D9.EnumAdapterModes(Adapter: LongWord; Format: TD3DFormat; Mode: LongWord; out pMode: TD3DDisplayMode): HResult; stdcall; begin {$IFDEF DEBUG} WriteToLog('C:\LeagueUtilityBox.log', 'TMyDirect3D9.EnumAdapterModes'); {$ENDIF} Result := fD3D^.EnumAdapterModes(Adapter, Format, Mode, pMode); end; function TMyDirect3D9.GetAdapterDisplayMode(Adapter: LongWord; out pMode: TD3DDisplayMode): HResult; stdcall; begin {$IFDEF DEBUG} WriteToLog('C:\LeagueUtilityBox.log', 'TMyDirect3D9.GetAdapterDisplayMode'); {$ENDIF} Result := fD3D^.GetAdapterDisplayMode(Adapter, pMode); end; function TMyDirect3D9.CheckDeviceType(Adapter: LongWord; CheckType: TD3DDevType; AdapterFormat, BackBufferFormat: TD3DFormat; Windowed: BOOL): HResult; stdcall; begin {$IFDEF DEBUG} WriteToLog('C:\LeagueUtilityBox.log', 'TMyDirect3D9.CheckDeviceType'); {$ENDIF} Result := fD3D^.CheckDeviceType(Adapter, CheckType, AdapterFormat, BackBufferFormat, Windowed); end; function TMyDirect3D9.CheckDeviceFormat(Adapter: LongWord; DeviceType: TD3DDevType; AdapterFormat: TD3DFormat; Usage: DWord; RType: TD3DResourceType; CheckFormat: TD3DFormat): HResult; stdcall; begin {$IFDEF DEBUG} WriteToLog('C:\LeagueUtilityBox.log', 'TMyDirect3D9.CheckDeviceFormat'); {$ENDIF} Result := fD3D^.CheckDeviceFormat(Adapter, DeviceType, AdapterFormat, Usage, RType, CheckFormat); end; function TMyDirect3D9.CheckDeviceMultiSampleType(Adapter: LongWord; DeviceType: TD3DDevType; SurfaceFormat: TD3DFormat; Windowed: BOOL; MultiSampleType: TD3DMultiSampleType; pQualityLevels: PDWORD): HResult; stdcall; begin {$IFDEF DEBUG} WriteToLog('C:\LeagueUtilityBox.log', 'TMyDirect3D9.CheckDeviceMultiSampleType'); {$ENDIF} Result := fD3D^.CheckDeviceMultiSampleType(Adapter, DeviceType, SurfaceFormat, Windowed, MultiSampleType, pQualityLevels); end; function TMyDirect3D9.CheckDepthStencilMatch(Adapter: LongWord; DeviceType: TD3DDevType; AdapterFormat, RenderTargetFormat, DepthStencilFormat: TD3DFormat): HResult; stdcall; begin {$IFDEF DEBUG} WriteToLog('C:\LeagueUtilityBox.log', 'TMyDirect3D9.CheckDepthStencilMatch'); {$ENDIF} Result := fD3D^.CheckDepthStencilMatch(Adapter, DeviceType, AdapterFormat, RenderTargetFormat, DepthStencilFormat); end; function TMyDirect3D9.CheckDeviceFormatConversion(Adapter: LongWord; DeviceType: TD3DDevType; SourceFormat, TargetFormat: TD3DFormat): HResult; stdcall; begin {$IFDEF DEBUG} WriteToLog('C:\LeagueUtilityBox.log', 'TMyDirect3D9.CheckDeviceFormatConversion'); {$ENDIF} Result := fD3D^.CheckDeviceFormatConversion(Adapter, DeviceType, SourceFormat, TargetFormat); end; function TMyDirect3D9.GetDeviceCaps(Adapter: LongWord; DeviceType: TD3DDevType; out pCaps: TD3DCaps9): HResult; stdcall; begin {$IFDEF DEBUG} WriteToLog('C:\LeagueUtilityBox.log', 'TMyDirect3D9.GetDeviceCaps'); {$ENDIF} Result := fD3D^.GetDeviceCaps(Adapter, DeviceType, pCaps); end; function TMyDirect3D9.GetAdapterMonitor(Adapter: LongWord): HMONITOR; stdcall; begin {$IFDEF DEBUG} WriteToLog('C:\LeagueUtilityBox.log', 'TMyDirect3D9.GetAdapterMonitor'); {$ENDIF} Result := fD3D^.GetAdapterMonitor(Adapter); end; function TMyDirect3D9.CreateDevice(Adapter: LongWord; DeviceType: TD3DDevType; hFocusWindow: HWND; BehaviorFlags: DWord; pPresentationParameters: PD3DPresentParameters; out ppReturnedDeviceInterface: IDirect3DDevice9): HResult; stdcall; var hr : HRESULT; begin {$IFDEF DEBUG} WriteToLog('C:\LeagueUtilityBox.log', 'TMyDirect3D9.CreateDevice'); {$ENDIF} hr := fD3D^.CreateDevice(Adapter, DeviceType, hFocusWindow, BehaviorFlags, pPresentationParameters, ppReturnedDeviceInterface); If Succeeded(hr) Then begin {$IFDEF DEBUG} WriteToLog('C:\LeagueUtilityBox.log', 'fD3D^.CreateDevice Succeeded'); {$ENDIF} ppReturnedDeviceInterface := TMyDirect3DDevice9.Create(PIDirect3D9(@Self), @ppReturnedDeviceInterface); end; Result := hr; end; end.
UPDATE: So, since Delphi interfaces seem to act differently than real ones (Delphi has an intermediate option for successfully communicating with other interfaces). So I just converted the interface to an array of pointers.
Now the program successfully calls CreateDevice (). I see this both in magazines and in Ollydbg.
Now it happens that when CreateDevice calls the original IDirect3D9.CreateDevice (), it will work again. When I debug in Ollydbg, I notice that this pranks the pointer once too much.
UPDATE 2: Ok, fixed some issues with the PIDirect3D9 and IDirect3D9 pointers in different places. Thus, the original IDirect3D9.CreateDevice () is called. But these are errors with D3DERR_INVALIDCALL !!
So confusing.
UPDATE 3: Well, with some extra debugging, it seems that when I call the function, the extra parameter is pushed onto the stack. This makes the first parameter invalid. This is further confirmed by DirectX Debugging, which states that the iAdapter parameter is invalid (first parameter).
UPDATE 4: Using IntRefToMethPtr () to get a direct pointer to the original call to CreateDevice, I was able to get it to call the stacks the same way. The same result. It seems like I went the wrong way, trying to plug it into Delphi.
UPDATE 5: Rewritten interception method. Now I just bind essentially EndScene (). The hook now works fine in the test program (Vertices.exe, which appeared with a demo of the hook found in the first URL in this post). But in the main game, he crashes the game. In any case, I learned a lot.