How to do 3D selection/picking using OpenGL

11
votes

I have some objects in the scene, some may occlude others. When I click the mouse or drag-select to get a selection rectangle, I want to select/pick only the objects that I can see from this perspective. The application currently uses GL_SELECT render mode but as we know, this selects occluded objects too. Also, I read that this is deprecated in OpenGL 3.

There are two methods that are currently appealing to me. First is Object Selection using the Back Buffer (Red book, chapter 14): setting the colour of each object to it's object id and reading the colour of pixels from the back frame buffer. The second is occlusion queries (superbible, 4th ed, chap 13).

Other approaches I have ruled out are looking at the min/max z values in the selection buffer and doing custom ray/object detection outside of GL.

I have some questions: 1) If GL_SELECT is deprecated in recent OpenGL, what alternatives are developers supposed to be using? 2) I've only ever read about occlusion queries being employed to speed up rendering. Can they be used for selection/picking, and are there drawbacks? 3) The existing application has a handful of glColorXXX calls. If I went the back buffer route, and used glColorMask(FALSE,FALSE,FALSE,FALSE), will this effectively turn the glColourXXX calls into calls that have no effect, thereby letting me control the colour in a single place when rendering in select mode? 4) Which route is best/canonical?

1
opengl
You really couldn't find this already asked and answered on SO? - Ben Voigt
@Ben: There are no questions in SO AFAICT looking at occlusion detection in OpenGL in the context of picking, and none that contrast it with the method that reads the colour of the clicked pixel. - PeteUK
Why have you ruled out "object detection outside of GL"? Only the simplest of graphical displays do not benefit from a spatial acceleration structure. - cmannett85
@cbamber85: Good question! I'm working on an existing application and to do the object detection outside of GL would require more changes so objects provided accurate bounding volumes. Secondly, I'm a novice when it comes to this (i.e. I'd never heard of a spatial acceleration structure until 3 minutes ago!). I will look into this, and in the fullness of time would like to incorporate a better method in my system, but for now I'd like to go OpenGL route. By the way, can you provide any pointers to learn more about the method you suggest? (The Wiki page on it is an orphan). - PeteUK
@cbamber85: Looks like I should be investigating Octrees, KD-trees, and the like? - PeteUK

1 Answers

4
votes

I decided to implement the selection using the back buffer. Here's my attempt to answer my questions:

If GL_SELECT is deprecated in recent OpenGL, what alternatives are developers supposed to be using?
I think it's best to not employ OpenGL to do this task but to use spatial acceleration structures as user chamber85 suggested in comments to the original question.

I've only ever read about occlusion queries being employed to speed up rendering. Can they be used for selection/picking, and are there drawbacks?
I'm sure they could but one would need to know all the objects they want to query for occlusion before the draw. Using back buffer and colour selection, one can just see what is under the cursor or within a rectangular region and filter from there.

The existing application has a handful of glColorXXX calls. If I went the back buffer route, and used glColorMask(FALSE,FALSE,FALSE,FALSE), will this effectively turn the glColorXXX calls into calls that have no effect, thereby letting me control the colour in a single place when rendering in select mode?
The answer is no. Calling glColorMask() with all GL_FALSE parameters will not mean that glColor3ub() calls (for example) will not be honoured. It simply specifies a filter/mask for colours just before they are written to the colour buffer. The original thought was to set the colour to the object id, then call glColorMask() to ignore all subsequent glColorXXX() calls. This strategy is doomed as the colour representing the object id would also be masked out.

4) Which route is best/canonical? I would say the back buffer colour selection is generally best as it doesn't require setting up the occlusion queries before/during the draw.