I'm new to opengl es and trying to figure out how to convert an x, y screen coordinate to its respective opengl world coordinate. All of the examples I've found so far use GLU.gluUnProject() to convert between the two coordinate systems however, I'm not sure this method is usable in opengl es 2.0 and if it is I'm not sure how to retrieve the projection and model view matrices. The other examples that use gluUnProject access the matrices through the use of a MatrixGrabber class which is opengl es 1.X specific. I've also found iphone examples that use a vertex shader to convert between the coordinate systems however, I haven't been able to get this technique to work either. It seems there would be a simple way to convert 2D screen coordinates to 2D world coordinates. Any help would be much appreciated. Thank you.
1 Answers
Your assumption is incorrect, there is no simple way to convert screen coordinates to world coordinates.
Given a 2d screen coordinate and an existing view and projection matrix there is no way of determining the correct z-coordinate besides sampling the depth buffer (provided you have a depth buffer).
Projecting a 2d point into a view frustum would only give you a ray (or a line segment from the point projected on the near clipping plane to the point projected on the far clipping plane).
The following may be besides the point as you are using java, but I'm going to mention it anyway: if you consider using c++ I suggest you checkout the glm library : http://glm.g-truc.net/
It provides replacements for deprecated glu functions, like glm::project and glm::unproject. It is an excellent replacement for the opengl matrix stack and glu functions.
I hope this helps answering your question. Good luck.
