The GDI library dates back to the early 90s and the first versions of Windows. It's supports a mix of hardware-accelerated raster ops and software vector graphics drawing. It is primary for drawing 'presentation' graphics and text for GUIs. GDI+ was a C++ class-based wrapper for GDI, but is basically the same technology. Modern versions of Windows still support legacy GDI APIs but modern graphics cards don't implement the old fixed-function 'raster ops' hardware accelerated paths--it's not really accurate to even refer to the video cards of the early 90s "GPUs".
OpenGL and Direct3D are graphics APIs built around pixels, lines, and triangles with texturing. Modern versions of both use programmable shader stages instead of fixed-function hardware to implement transform & lighting, texture blending, etc. These enable 3D hardware acceleration, but by themselves do not support classic 'raster ops' or vector-based drawing like styled/wide lines, filled ellipses, etc. Direct2D is a vector graphics library that does these old "GDI-style" operations on top of Direct3D with a mix of software and hardware-accelerated paths, and is what is used by the modern Windows GUI in combination with DirectWrite which implements high-quality text rendering.
See Graphics APIs in Windows, Vector graphics, and Raster graphics.
In short, for modern Windows programming you should use Direct2D/DirectWrite for 'presentation graphics' perhaps via a wrapper like Win2D, and Direct3D for 2D raster "sprite" graphics or 3D graphics rendering (see DirectX Tool Kit).