The "CUDA c++" language is a c++ derived language but it is not supported by standard compilers, but can be compiled by nVidia's nvcc compiler. This can leads to multi-language projects, which have their GPU modules compiled with nvcc and the non-GPU modules compiled with gcc.
For example, the syntax kernel<<<dims>>>(params) is not a part of c++.
Does nVidia include a GPU library (as part of CUDA) that actually can be used with a standard c++ compiler such as gcc?
CUDA GPUs require device code to be compiled for the GPU, which is not the same as x86 machine code that you would find in a typical library compiled for an x86 target.
So, to use a GPU, somehow you are going to need the nvidia compiler to generate machine code.
There are various ways to accomplish this:
Using libraries. Libraries that ship with CUDA such as CUBLAS, CUSPARSE, CUSOLVER, and CUFFT include precompiled GPU routines that are callable and linkable from ordinary host code. These libraries can all be used with a standard c++ compiler such as g++.
Most routines from the cuda driver API and cuda runtime API are callable and linkable from ordinary host code, using a standard c++ compiler. However this doesn't address the question of what to do with device code. If device code is delivered in PTX form using the driver API (which does not include syntactical elements like kernel<<<...>>>), then nvcc is not required, and the application can be completely built using a standard compiler such as g++.
If device code is delivered in C/C++ source form (i.e. CUDA C/C++), then ordinarily nvcc will be needed to either convert the code to PTX (so that method 2 above can be used) or to convert directly to a binary compiled format, which is usable by either the driver API or the runtime API.
In the future, you may also want to keep an eye on evolving capabilities of the CUDA runtime compilation mechanism.
Since you mention gcc, you may also want to keep an eye on the evolving capabilities of OpenACC support in gcc.
