X86: ZF is not always updated AND?

Question

I am debugging my x86 code and the problem is being tracked with the AND instruction sometimes it doesn’t clear the ZF flag if the result is not zero. Here is the problematic part of the code:

 0257A9F9 mov edx, dword ptr [ecx + 18h] 
 0257A9FC and edx, 80000h 
 0257AA02 int 3    
 0257AA03 je 0257AA2A 

I added a breakpoint after AND for debugging. When it stops at the breakpoint EDX == 0x80000 and ZF == 1. But ZF should be cleared if EDX! = 0. The code works fine when there is one step in the debugger, but it does not work sequentially during normal run.

Here is a screenshot of a debugger session.

Any clues?

If it matters, the code is generated by JIT, so I am executing the data.

Thanks in advance.

Answer 1

Thanks to everyone. It was my mistake, sorry to bother you. There is a branch "int 3" from another place. That is why flags do not follow the instructions before "int 3". I was confused, always having edx == 0x80000 at that moment. Sorry again.

Answer 2

You can easily examine the int 3 handler to see if it returns using iret (i.e., the flags of the callers pops up) or if it returns using retf 2 (i.e. saves the flags from the handler).

Answer 3

According to the Intel instruction set instruction, ZF is always set according to the result. Can something in the int 3 handler manipulate this?

Edit: after further searching the manuals (thank god Intel sends free copies!), My only ideas are that it is either an int 3 handler, or its configuration, or a processor that considers only dx instead of edx when setting flags. Both seem unlikely, but the latter seems completely implausible. What mode are you working in? (Real, protected, unreal, long?)

Answer 4

Perhaps your debugger is doing something special, for example, synchronizing memeroy and registers. When you run it without a debugger, is that not so?