Detect the byte that appears during debugging after a long manually encoded call

Question

I couldn’t get MASM to accept the remote call statement written as call 0f000h:1260h , possibly due to problems that arose in this question .

Instead of messing with cryptic MASM directives, I decided to manually encode it in my program using DB as follows:

 pushf ;IRET will be executed, push flags. db 9ah,60h,12h,0,0f0h ;call location f000:1260. ;Location pointed to by int 1c (System timer tick) ;BIOS defaults it to a dummy IRET 

Tracing the program through DEBUG.COM, I noticed that after executing the call command the message "DB FE" appears. However, this does not happen when executing int 1ch . What is the difference between the two methods of jumping to f000: 1260?

I assumed that DEBUG would not recognize 0xfe (along with the following bytes) as a valid opcode. I reset the location f000: 1260 to see what bytes were there.

The 0xfe byte is really present along with some other bytes. I know that 0xcf itself is the opcode for IRET (which is all I expected to find), so what are these other bytes?

Here's the IVT record for int 1ch at point 0000: 0070.

UPDATE

As Michael Patch stated in his answer, weird bytes make up the callback mechanism in DOSBox. I was curious to know what would happen if I try to execute this callback in my main program.

Performance:

 xor ah, ah ;select set video mode function mov al, 13h ;320x200 256 colors db 0feh,38h,18h,00h ;set video mode DOSBox callback. ;Nothing pushed to stack. 

This seems to be exactly the same as doing:

 xor ah, ah ;select set video mode function mov al, 13h ;320x200 256 colors int 10h ;set video mode. ;Three registers pushed, FLAGS altered (by INT) ;callback occurs, registers popped (by IRET) 

The only difference is that int presses FLAGS , CS and IP , and also clears IF and TF. The program returns to IRET (at f000: 1264), which cancels all this.

"DB FE" is still displayed in DEBUG. I think the callback is only called by a combination of 0xfe and 0x38. First, he tries to execute the 0xfe command, which in this case is not part of the actual operation code and does nothing (0xfe is part of the inc operation code if valid bytes follow), and then, when the next 0x38 occurs, a callback occurs.

Answer 1

0xFE 0x38 not a specific prefix and / or instruction on a real Intel x86 processor. In DOSBox, the 0xFE 0x38 is a special 4-byte instruction. The remaining two bytes make up a 16-bit value, which acts as a DOSBox callback index index. In this case, the index is 0x0013 .

What this effectively does is make calls through DOSBox itself to complete the requested task. DOSBox will perform any necessary processing, set the registers inside the emulator and then return. The next instruction is IRET (0xCF). This will stop the interrupt and continue processing instructions before calling the interrupt.

I discovered this by looking at the DOSBox code. In particular, the CALLBACK_SetupExtra function:

 case CB_IRET: if (use_cb) { phys_writeb(physAddress+0x00,(Bit8u)0xFE); //GRP 4 phys_writeb(physAddress+0x01,(Bit8u)0x38); //Extra Callback instruction phys_writew(physAddress+0x02,(Bit16u)callback); //The immediate word physAddress+=4; } phys_writeb(physAddress+0x00,(Bit8u)0xCF); //An IRET Instruction 

This code sets up DOSBox callbacks where an IRET is required after the callback.