Is there a way to increment all fields in a Redis hash at once?

Question

I want to use Redis to cache a large amount of highly dynamic counters. In my case users are subscribing to sources and I want to cache each user's counter for that source. When a new item arrives at the source it's natural that the counters for all users subscribed to this source should be incremented by 1. Some sources have hundreds of thousands of subscribers, so it's important that this happens immediately.

However, Redis doesn't have a native method to increment all hash fields at once, only HINCRBY. Is there a solution for this?

While searching, I stumbled upon some threads where people wanted a bulk version of HINCRBY, but my case is different. I want to increment all fields in the hash.

Did you try working with LUA? redis.io/commands/eval redis.io/commands/evalsha — Guy Korland
Looks like a promising lead, though I'm a bit concerned about performance. I will probably call this 50-100 times per seconds and from what I read, lua scripts block the whole server until they are executed, so this could lead to contention. — Jacket
Are you calling all these 100 times in one call? If not it shouldn't impact too much, you should try it — Guy Korland
No, they could all be coming in parallel from different processes, even different servers. At the same time I need redis to be able to respond as fast as possible to clients that wants to fetch counters. I expect no less than 5000 gets per second from clients, probably above 10000. — Jacket
Single redis should be easily hold such numbers if for some reason it doesn't you might want to consider clustering (see: redis.io/topics/cluster-tutorial or redislabs.com/redis-enterprise) — Guy Korland

Guy Korland Guy Korland · Accepted Answer · 2019-08-05T07:12:45

Such operation should be achieved by using LUA script see:

You can use LUA script to "batch" couple of operations on a single hash

Is there a way to increment all fields in a Redis hash at once?

1 Answers