Understanding sign and overflow flag in assembly

5
votes

This question is about the cmp instruction in assembly. I cannot understand how my books reasoning regarding the SF and OF flags. 

 cmp vleft, vright

According to my book: For signed integers, there are three flags that are important: the zero (ZF) flag, the overflow (OF) flag and the sign (SF) flag. The overflow flag is set if the result of an operation overflows (or underflows). The sign flag is set if the result of an operation is negative. If vleft = vright, the ZF is set (just as for unsigned integers). If vleft > vright, ZF is unset and SF = OF. If vleft < vright, ZF is unset and SF != OF. Do not forget that other instructions can also change the FLAGS register, not just CMP.

First, let's consider the vleft > vright case. My book says the following:

Why does SF = OF if vleft > vright? If there is no overflow, then the difference will have the correct value and must be non-negative. Thus, SF = OF = 0. However, if there is an overflow, the difference will not have the correct value (and in fact will be negative). Thus, SF = OF = 1.

The first part i understand that SF = OF = 0. It could for example be:

0111 - 0101 = 0111 + 1010 + 1 = 10010 = 7 - 5 = 2

This would not set the OF or SF flag.

The case could also be:

1100 - 0101 = 1100 + 1010 + 1 = 10111 = -4 - 5 = 7 (-9 if we had more bits)

This would not set the SF flag (since the answer is the positive 7) but would set OF = 1, thus SF != OF. This clearly goes against the explanation of my book which says they should be equal.

What am I missing here?

Thank you!

2
assemblyx86
It's weird to talk about assembly and processor details such as flags and such without mentioning which CPU you're talking about. There are a few, you know. – unwind
Sorry about that, but this should be X86 (80386 and later) processors. – Lukas Arvidsson

2 Answers

6
votes

You are gonna bang your head on the wall, sorry about that :)

In your example, -4 is not greater than 5! So yeah, OF will be 1 and SF will be 0, and they won't be equal, and that means -4 is less than 5, and that's correct.

To illustrate the SF = OF = 1 case reverse the operands: check if 5 > -4 by doing

5 - (-4) = 5 + 4 = 1001b = -7

2
votes

Your book is right about OF== (SF!=CF) when both operands have the same sign. When the operands are of different sign, there can be no OF.

However, in RTL level, the OF flag is most often computed as difference of carry_in != carry_out calculated at the sign bit.

i.e. for 1100-0101 = 1100+1010+1
bit position 0:  c_in=c_0= 1, a_0 = 0, b_0 = 0;  result=1, c_out=0
             1:  c_in=c_1= 0, a_1 = 0, b_1 = 1;  result=1, c_out=0
             2:  c_in=c_2= 0, a_2 = 1, b_2 = 0;  result=1, c_out=0
             3:  c_in=c_3= 0, a_3 = 1, b_3 = 1;  result=0, c_out=1

Here c_in_3 != c_out_3, which means an overflow.