Assembly - use of carry and overflow flags in actual assembly programming

0
votes

After some reading on http://teaching.idallen.com/dat2343/10f/notes/040_overflow.txt, I understand that carry flag is used to indicate some "data loss"/"out of bounds" and overflow flag is used to indicate "most significant bit is on/off whereas they were off/on previously". I also saw another answer that gives a tip when checking flags for signed/unsigned numbers:

"check for negative and overflow flags when dealing with signed numbers and carry flags with unsigned." (apologies as I can't find that answer again for reference)

I understand the flags' associations, but as far as I've read, the cases for flag checking examples only indicate when the flags are set, and I don't really understand the carry flag and overflow flag's use beyond error checking.

For example the flags for ja and jg, I read an answer that stated:

ja jumps if CF = 0 and ZF = 0 jg jumps if ZF = 0 and SF = OF

(Difference between JA and JG in assembly)

I understand what the zero flag is used for (to check that they are not equal) , but not so much on CF and SF/OF.

Below is my stab at the interpretation, disregarding the ZF as a pre-requisite

ja (unsigned): If CF = 0, there is no "borrow out" from the first operand for the subtraction. Hence if ZF = 0, the comparison is true.

jg (signed): If SF (we call it negative flag I believe) and OF (we call it the oVerflow flag) are both 0, that means both operands are positive and result is also positive, indicating lhs >= rhs ; can't wrap my head around the cases of when they are both 1 or SF != OF, as I do not see how this indicates lhs >= rhs and lhs < rhs.

Edit: I kept testing cases for when SF = 1 and OF = 1, but I've only encountered cases where they both equal 0. Hence I don't understand why SF = OF works as a comparison. For example, 1000 - 1000, 1000 - 0100. Or am I misunderstanding the intention of this comparison that it will never hit SF = 1 and OF = 1, only if they do not equal to each other, the result will be false?

Thank you for taking the time to read this and any pointers as to how the flags are used in this example, how flags are used in common assembly code would be greatly appreciated. Let me know if I'm missing any information needed.

1
assemblyx86overflowflagscarryflag
Try 20 - 10, 10 - 20, -20 - 10, -10 - -20, -10 - 20, -125 - 10, 125 - -10, etc. The patterns should become clear. (Do them as byte-sized operations; the last two will overflow.) – prl
Thank you, so the last two, I can see that they overflow and sets the negative bit, so that cleared it up for me, but it doesn't make sense to me in decimal format (i.e. -125 >= 10 (?) ). Maybe I'm misunderstanding something here, but can you elaborate? – RosaryLightning X
For -125 - 10, SF=0 and OF=1, so -125 is less than 10. For 125 - -10, SF=1 and OF=1, so 125 > -10. – prl

1 Answers

1
votes

The "condition" mnemonics (like A for Above) work if flags are set by cmp eax, edx.

e.g. after cmp edx,eax, the Above condition is true if edx > eax (unsigned compare),
and the Greater condition is true if edx > eax (signed compare).

Look at the definitions of each condition in http://felixcloutier.com/x86/Jcc.html or http://felixcloutier.com/x86/SETcc.html and work through it yourself if you're curious.

Checking for "data loss" or overflow is one use-case for flags (and where their names come from), but checking them after a compare is far more common.