X86 - illegal 0xff / 7 code for Windows

Question

I am currently developing an x86 disassembler, and I started disassembling the win32 PE file. Most disassembled code looks good, but there are some cases of illegal operation code 0xff / 7 (/ 7 means reg = 111, 0xff is the group of opcodes inc / dec / call / callf / jmp / jmpf / push / illegal with operand g / m 16 / 32). The first assumption was that / 7 is a pop command, but it is encoded by 0x8f / 0. I checked this against the official Intel Architecture Volume 2: Reference Set Reference software development guide - so I'm not just skipped.

Disassembly example: (S0000O0040683a is lability bounded by another instruction)

S0000O0040683a: inc edi ; 0000:0040683a ff c7 test dword ptr [eax+0xff],edi ; 0000:0040683c 85 78 ff 0xff/7 edi ; 0000:0040683f ff ff 

BTW: gdb parses this equally (except for the 0xff error not inferior to -1 in my disassembly):

 (gdb) disassemble 0x0040683a 0x00406840 Dump of assembler code from 0x40683a to 0x406840: 0x0040683a: inc %edi 0x0040683c: test %edi,0xffffffff(%eax) 0x0040683f: (bad) End of assembler dump. 

So, the question is, is there any default handler in the exception handler for the Windows illegal operation code handler that implements any functions of this illegal operation code, and if so: what happens there?

Regards, Bodo

Answer 1

After many extra hours getting my disassembler to output the output in the same syntax as gdb, I could distinguish between the two versions. This showed a rather uncomfortable error in my debass: I forgot to take into account that the transition command 0x0f 0x8x has a binary opcode (plus operand rel16 / 32). Thus, each transition target 0x0f 0x8x was disabled by one, leading to a code that is actually inaccessible. After fixing this error, the code codes 0xff / 7 are no longer parsed.

Thanks to everyone who answers my question (and commenting on these answers), and therefore, at least, trying to help me.

Answer 2

Visual Studio parses this with the following:

 00417000 FF C7 inc edi 00417002 85 78 FF test dword ptr [eax-1],edi 00417005 ?? db ffh 00417006 FF 00 inc dword ptr [eax] 

Obviously, a general security error occurs in 00417002, because eax does not indicate anything significant, but even if I turn it off (90 90 90), it throws an illegal exception for the operation code in 00417005 (it is not processed by the kernel). I am sure that this is some information, not executable code.

Answer 3

To answer your question, Windows will close the application with exception code 0xC000001D STATUS_ILLEGAL_INSTRUCTION . The dialog box will correspond to the dialog box used for any other application crashes, regardless of whether it offers a debugger or sends an error report.

Regarding the code provided, it was apparently either improperly assembled (encoding a more than 8-bit offset) or actually data (as others have suggested). A.

Answer 4

It looks like 0xFFFFFFFF was inserted instead of 0xFF for the test command, probably by mistake?

85 = test r / m32, and 78 is a byte for the parameters [eax + disp8], edi, while disp8 should be specified, which should be 0xFF (-1), but as a 32-bit signed integer, it is 0xFFFFFFFF.

So, I assume that you have 85 78 FF FF FF FF, where should there be 85 B8 FF FF FF FF for 32-bit offset or 85 78 FF for 8-bit offset? If so, the next byte in the code should be 0xFF ...

Of course, as already mentioned, this can only be data, and do not forget that the data can be stored in PE files, and there is no reliable guarantee of any particular structure. In fact, you can embed user-defined code or data in some header fields of MZ or PE if you aggressively optimize the .exe size.

EDIT: in accordance with the comments below, I also recommend using an executable file where you already know exactly what the expected disassembled code should be.