How to efficiently pass a variable from Matlab to GPU in Asynchronous way?

Question

In my CUDA project, i could define a pinned memory, copy the data from a .txt file to the pinned memory and use streaming to copy the data to GPU while doing my processing in Kernel. Now, I want to make a CUDA MEX file and pass my data (called RfData) variable to it in Matlab. However, i have noticed that there is no way of allocating MATLAB arrays directly as pinned CUDA memory. So, i have to do it with the pageable memory as follows:

int* RfData;
RfData = (int *)mxGetPr(prhs[0]); 
int* Device_RfData;  
int ArrayByteSize_RfData = sizeof(int) * (96* 96* 2048);
cudaMalloc((int**)&Device_RfData, ArrayByteSize_RfData);
cudaMemcpy(Device_RfData, RfData, ArrayByteSize_RfData, cudaMemcpyHostToDevice);

this is important for me to copy the RfData asynch with streaming. The only way i could think of was to first copy my RfData to a pinned memory and then use streaming:

 int* RfData_Pinned;
cudaHostAlloc((void**)&RfData_Pinned, ArrayByteSize_RfData,  cudaHostAllocWriteCombined);
for (int j = 0; j < (96* 96* 2048); j++)
{
    RfData_Pinned[j] = RfData[j];
}

However, it increased the overal processing time of my MEX function.

SO, the question is now: how can i transfer my data from matlab to GPU in asynch way? Maybe there is a command in CUDA which allows fast copy of data from pageable to pinned memory!!!?

Thanks in advance, Moein.

Ander Biguri Ander Biguri · Accepted Answer · 2021-03-26T18:40:19

You can indeed allocate pinned memory with cudaHostAlloc but if you have already allocated memory, you can instead just pin it with cudaHostRegister, that takes an already allocated host array pointer (taken from mxGetPr in your case).

Note that this will take time to pin the memory, but possibly less tha doing cudaHostAlloc and then copying it.

How to efficiently pass a variable from Matlab to GPU in Asynchronous way?

1 Answers