Is it possible to synchronize OpenCL work-groups?
For example, I have 100 work-groups every work-groups have only one item (don't ask me why, this is an example), and I need to put barrier to every work-item which ensure that all work-groups will be continue after every work-item in this 100 work-groups reaches this barrier point.
No, you can't. You can synchronize threads inside a group, and you can synchronize kernel executions inside a command queue.
You may be able to synchronize a small number of groups as long as they are all executed simutaneously, using atomic accesses. But it will freeze if some groups are scheduled later, and you have no control on this.
In a word, no you can't. The OpenCL paradigm is a data parallel one where workgroups are intended to be independent. The only workgroup scope synchronization mechanism is at the command queue level, ie. separate kernel launches. If you algorithm can't accommodate that, you either need a new algorithm, or use a different programming model.
You need to keep in mind that there are often far more workgroups than hardware to execute them simultaneously. Synchronization in such cases is impossible. There are ways to implement a spinlock or critical section across a hardware dependent number of work groups using atomic memory access primitives, however they are really an abuse of the programming model and tend to only be useful where there is relatively little interaction between workgroups.
