throttle consumer in producer/consumer threads

Question

There is a single consumer and single producer thread. The producer thread data acquisition is slow. It queries a socket for data and the time it takes to produce data for the consumer is significantly longer than the time it takes for the consumer to process and send the data out. The problem is I am updating a display so I want the updates to slow down so they appear continuous rather than update in bursts.

I am using a double buffer right now but the consumer is waiting too long for the buffers to be swapped because the producer is taking too long to produce data. Perhaps if I slice up the data into smaller blocks and use a queue instead? That way the producer would feed the consumer a little at a time? Has anyone ever run into this problem?

Why is updating the display immediately an issue? Doesn't that make your application seem slower than it actually is? — fbrereto
Yes. It improves the user experience. Right now it updates really fast then stops because it needs to fetch more data. I want to spread the time out so there are no stops. Is there another simpler way I can achieve this? — eat_a_lemon
Artificially throttling down a consumer thread kind of defies the use of threads, to be honest. The main reason why one uses threads is that things seem to go faster (and in the case of multiprocessing actually do go faster) because the consumer does not block while the producer is reading. If you throttle down the consumer, you could as well scratch the entire multithreading and just write to the screen whatever comes over the wire as it comes. — Damon
There was a question not long ago about smoothing out download progress bars, but I'm too lazy to search for it. — ninjalj
If I write to the screen as it comes then it abruptly stops when it runs out of data. I need to spread the time out for each iteration that is sent to the screen. I am collecting frames of data but I want the frames to appear continuous even though they are not. This will slow down the update rate but that is alright. — eat_a_lemon

johnnycrash johnnycrash · Accepted Answer · 2011-04-05T19:08:06

Why not have a thread that updates the screen once a sec? The thread can sleep for a sec, wake up, check into what the producer and the consumer are doing, and update the screen based on their progress. You would get updates every second. If you want them faster or slower, then change the timer interval.

throttle consumer in producer/consumer threads

2 Answers