What I found (VC8 specific probably, although it may work on other compilers):
- The vtable pointer is held in the first 4 bytes of a class with virtual functions. That means you can cast the class to a size_t pointer, and dereference it to get the address of the vtable as a size_t.
The vtable is an array of addresses, one entry per virtual function and is ordered in by the order the functions appear in the class. I'm not sure where a virtual destructor comes in, I didn't bother checking - but I expect it's the same as a normal func (Position defined by the location in the class header). - The vtable is write protected, so you need to use VirtualProtectEx to enable write access on it.
- You need to use a naked stub function to call a "normal" C function to get around the calling convention crap.
Overall, nothing particularly mind blowing. Anyway, here's the code I ended up with (To hook the SetRenderState() call):
HRESULT MySetRenderState(D3DRENDERSTATETYPE State, DWORD Value){ // Do whatever you want here}__declspec(naked) HRESULT stub(D3DRENDERSTATETYPE State, DWORD Value){ __asm { push ebp; mov ebp, esp; sub esp, 0Ch; } MySetRenderState(State, Value); __asm { mov esp, ebp; pop ebp; ret 0Ch; }}// Grab the vtable and remove write protection on itsize_t nEntry = 57;size_t* pVTable = (size_t*)*(size_t*)m_pDevice;DWORD dwOldProt;VirtualProtectEx(GetCurrentProcess(), &pVTable[nEntry], sizeof(size_t), PAGE_EXECUTE_READWRITE, &dwOldProt);// Thunk the vtable so SetRenderState() calls "stub" insteadpVTable[nEntry] = (size_t)&stub// Test the new callm_pDevice->SetRenderState(D3DRS_ZENABLE, 1);Joy joy joy. Anyway, back to work for me...
