I'm attempting to optimize an OpenCL matrix multiplication program for an NVIDIA GeForce 9800 GT graphics card. I'm multiplying two 512x512 matrices, but I'm not sure what global and local work group sizes I should use. Can anyone point me in the right direction?
2
votes
Global size should ideally be as high as possible insofar as you do not need to interrupt the computation to add or modify inputs, if this is so then there are no diminishing returns to use as large a global size as you need. Though in practice, if you are using a GPU, most of them don't support task concurrency yet, so a GPU running your kernels will do only that. So if you were using it as a main graphics card, your screen will freeze. As for local work size, it depends on the kernel and the device, you can get OpenCL to try and optimize it (look up clGetKernelWorkGroupInfo)
- Thomas
Keep in mind that Nvidia GPUs group threads into groups of 32 threads (a warp), hence for coalesced memory access, you would want to have one dimension of the local work group size be a multiple of 32.
- chippies
3 Answers
3
votes
3
votes
As 512 is a power of two, you can try the following sizes:
size_t global_work_size[2] = {512, 512};
size_t local_work_size[2] = {2^p, 2^n};
where :
(2^m + 2^n) % 32 = 0:32must be a divider of your number of threads, as a wrap will contain32threadsThe power of two is need as the local work size must be a divider of the global work size.
With a size of
512, it will certainly not be a problem, but you have to take into account theCL_DEVICE_MAX_WORK_GROUP_SIZE(given byclGetDeviceInfo()) parameter. For bigger matrices, you will have to use more than2dimensions.The number of dimensions is itself limited by
CL_DEVICE_MAX_WORK_ITEM_DIMENSIONS...
Indeed the best choice is algorithm dependent, and depends on the kind of matrix multiplication you intend to perform.
1
votes
Agree with Dithermaster, and I want to add that the peak performance is highly platform-dependent, even the GPUs from the same vendor may have different optimal work group sizes if the GPUs are not the same model/generation.
To get the peak performance, you really need to experiment (pre-execution training) on the target platform and find the optimal configuration. Remember that not only the the work group size but also the work group shape could affect the performance significantly. That being said, you really need to try all the combinations as below. Suppose the max work group size for the kernel is 1024, which means for a 2-D work group, you can have the following combinations: (1, 1024), (2, 512), (4, 256), (8, 128), (16, 64), (32, 32), (64, 16), (128, 8), (256, 4), (512, 2), and (1024, 1).
Be careful that (1, 1024) and (1024, 1) could lead to totally different performance, due to the memory architecture, cache architecture and the way the wave is schedule and so on.
Yes, of course, you also need to consider wave/warp size as well as the coalesced memory access. Here, I am just talking about a general OpenCL workload, especially when the workload does not show any structural pattern, it would be good to experiment with as many as combinations to make sure you don't miss anything.
