AVX/SSE round floats down and return vector of ints?

Question

Is there a way using AVX/SSE to take a vector of floats, round-down and produce a vector of ints? All the floor intrinsic methods seem to produce a final vector of floating point, which is odd because rounding produces an integer!

Conversion to integer must round or truncate somehow, but you can round an FP value to the nearest integer without converting. (See float nearbyintf(float x)) — Peter Cordes
Definition of round() - stackoverflow.com/questions/3597197. — Royi
@Royi: can you please delete your misleading comment? There are no x86 intrinsics for the C round() rounding mode, which rounds +-0.5 away from zero. x86 rounding intrinsics that use round-to-nearest use the IEEE default rounding mode, banker's rounding (nearest-even as a tiebreak.) rint / lrint / nearbyint all give you the default rounding mode, and are faster than round() on x86 (especially with SSE4.1 or AVX, or any time when you're converting to integer). So linking to a definition of round() is off topic. — Peter Cordes
@PeterCordes, I linked to the answer which defines round() as in Wikipedia. I didn't get your comment, does round() in C obeys this definition or not? — Royi
This question is about rounding in general, no the C round() function. And specifically rounding down (floor or trunc). Nowhere does the question say anything about rounding functions that use the rounding mode of the C round() function. It's not the default IEEE rounding mode or anything. It's maybe interesting to mention it in an answer as an alternative to _mm_round_ps(x, _MM_FROUND_TO_NEAREST_INT |_MM_FROUND_NO_EXC), if you're going to talk about other rounding modes. — Peter Cordes

zx485 zx485 · Accepted Answer · 2016-05-07T17:42:55

SSE has conversion from FP to integer with your choice of truncation (towards zero) or the current rounding mode (normally the IEEE default mode, nearest with tiebreaks rounding to even. Like nearbyint(), unlike round() where the tiebreak is away-from-0. If you need that rounding mode on x86, you have to emulate it, perhaps with truncate as a building block.)

The relevant instructions are CVTPS2DQ and CVTTPS2DQ to convert packed single-precision floats to signed doubleword integers. The version with the extra T in the mnemonic does Truncation instead of the current rounding mode.

; xmm0 is assumed to be packed float input vector
cvttps2dq xmm0, xmm0
; xmm0 now contains the (rounded) packed integer vector

Or with intrinsics, __m128i _mm_cvt[t]ps_epi32(__m128 a)

For the other two rounding modes x86 provides in hardware, floor (toward -Inf) and ceil (toward +Inf), a simple way would be using this SSE4.1/AVX ROUNDPS instruction before converting to integer.

The code would look like this:

roundps  xmm0, xmm0, 1    ; nearest=0, floor=1,  ceil=2, trunc=3
cvtps2dq xmm0, xmm0       ; or cvttps2dq, doesn't matter
; xmm0 now contains the floored packed integer vector

For AVX ymm vectors prefix the instructions with 'V' and change the xmm's to ymm's.

ROUNDPS works like this

Round packed single precision floating-point values in xmm2/m128 and place the result in xmm1. The rounding mode is determined by imm8.

the rounding mode (the immediate/the third operand) can have the following values (taken from table 4-15 - Rounding Modes and Encoding of Rounding Control (RC) Field of the current Intel Docs):

Rounding Mode               RC Field Setting   Description
----------------------------------------------------------
Round to nearest (even)     00B                Rounded result is the closest to the infinitely precise result. If two values are equally close, the result is nearest (even) the even value (i.e., the integer value with the least-significant bit of zero).
Round down (toward −∞)      01B                Rounded result is closest to but no greater than the infinitely precise result.
Round up (toward +∞)        10B                Rounded result is closest to but no less than the infinitely precise result.
Round toward 0 (truncate)   11B                Rounded result is closest to but no greater in absolute value than the infinitely precise result.

The probable reason why the return vector of the rounding operation is float and not int may be that in this way the further operations could always be float operations (on rounded values) and a conversion to int would be trivial as shown.

The corresponding intrinsics are found in the referenced docs. An example of transforming the above code to intrinsics (which depend on the Rounding Control (RC) Field) is:

__m128 dst = _mm_cvtps_epi32( _mm_floor_ps(__m128 src) );

AVX/SSE round floats down and return vector of ints?

3 Answers