Windows: Yes
X11/GLX: Yes
MacOS-X: It's complicated.
First the easy stuff: In Windows and X11 there are no such things as special fullscreen mode OpenGL windows. They're all just regular toplevel windows and you can add or remove the window decorations (title bar, border) anytime you like. If you remove the window decorations and set the window size to maximized you essentially get a fullscreen OpenGL window; in fact the graphics drivers are smart enough to detect this situation and switch to a fast track then.
Now MacOS X. In MacOS X clear distinction is made between Windowed and Fullscreen (which IMHO is quite annoying).
The good news is, that you can get access to the underlying context object which allows to implement some resource sharing so that you don't have to recreate the data containing OpenGL objects.
Big Fat Disclaimer: I never ventured as deep into MacOS X as I did with other OSs so my practical experience with this certain topic on MacOS X is only theoretical.
SDL2
to do this.SDL1
was a bit of a nightmare getting full screen working. Have no experience with mac though. – jozxyqk