redis lua bit overflow

Question

I'm working with redis lua and need to perform bitwise logic operations on a field up to 53 bits(the default length of integer part of redis ordered set score)

But it seems I'm out of luck:

127.0.0.1:6379> eval 'return bit.lshift(1, 30) ' 0
(integer) 1073741824
127.0.0.1:6379> eval 'return bit.lshift(1, 31) ' 0
(integer) -2147483648

It seems bit.* can operate only on 30 bits and then overflows(32 bit signed integer)

I'm using Linux 64 bits and redis is compiled for 64 bits as well. It looks like a limitation of bit library:

http://bitop.luajit.org/api.html

Note that all bit operations return signed 32 bit numbers (rationale). And these print as signed decimal numbers by default.

On the other hand...

eval 'return math.pow(2, 53) ' 0
(integer) 9007199254740992

Any idea how better to overcome this problem?

P.S. someone would say move this logic to client - but I can't. The piece is pretty complicated and needs to work closely to data

Ok, I found this one: stackoverflow.com/a/7333595/1812225 Purely awful and slow, but looks like redis leaves me without choice :( — let4be
For those interested adding link to redis issue: github.com/antirez/redis/issues/2843 — let4be

Diego Pino Diego Pino · Accepted Answer · 2015-11-02T20:53:50

It seems bit.* can operate only on 30 bits and then overflows(32 bit signed integer)

Not really. LuaJIT's BitOp works on 32-bit signed integers. That's the reason why 2^31 is a negative number. BitOp documentation explains that the reason to work with signed int32 and not unsigned is because of architecture compatibility issues:

Defining the result type as an unsigned number would not be cross-platform safe. All bit operations are thus defined to return results in the range of signed 32 bit numbers

http://bitop.luajit.org/semantics.html

This can be troublesome sometimes when comparing the result of a bit operation to a constant. In that case it's necessary to normalize the constant value using bit.tobit(). Example:

> = bit.lshift(1, 31) == 2147483648
false
> = bit.lshift(1, 31) == bit.tobit(2147483648)
true

In any case, LuaJIT's BitOp module is limited to 32-bit integer numbers.

On the other hand, if all the bitwise operations you need are lshift and rshift, it's possible to code those functions in plain Lua:

local function lshift(n, b)
   return n * 2^b
end

local function rshift(n, b)
   return n / 2^b
end

redis lua bit overflow

1 Answers