Black Screen when rendering with Oculus SDK

Question

I'm currently working on a basic Oculus Rift application based on the SDK 0.3.2 and OpenGL 3.2.0, to get me used to 3D engine and Oculus Rift technology.

For now I am trying to render a rotating cube inside the Oculus. For distortion, I decided to use the rendering engine provided by the Oculus Rift team in the SDK. So, first I render my scene in a single texture containing two copies of the scene with a different background color for visualisation (for now I'm not bothering with the stereoscopic aspect), and I know for a fact that it works:

The window is too big for it to be displayed completely on the screen, but we can clearly see the same scene. The window is cut on its center

---

Edit 4 (Final) :

After a lot of trial and error, and the advices of Jherico, I managed to make everything run.

It appear that, reagrding to Edit 3, I had to re-bind the vertex buffer glBindBuffer(GL_ARRAY_BUFFER, vertexBuffer); every frame. The code (without stereo or movement tracking) can be found here. Nota: the mode MODE_OCULUS_DEBUG_ does not work anymore. Aaaand the image is flipped.

---

Edit 3 :

I skipped to SDK 0.4.1, and followed all advices of Jherico (by the way, huge thanks), and I end up with this. I really don't know what's going on. But one thing I noticed is that I sometimes have to bind the frame buffer using glBindBuffer and sometimes glBindFramebuffer... Keep in minf that the texture is still exaclty the same as the beginning, on the first screenshot. I feel like this is an issue regarding frame timing. If I trigger the process of the half of the first frame quickly enough, I have no bug on the first frame. Note that I was trigger half a render every time, which means I had to trigger two times to get a frame. Then the second is always the same as the first one, the third can be glithcy if I trigger it to late, and then it always glitches after the fourth frame. The glitch appears only once, I couldn't make it appear two times, even if I wait for a long time.

I'll try to investigate that tomorrow, but if you have any idea, or if it is a common OpenGL bug or misunderstanding, you're welcome to help :)

You can find the code here

---

*Edit 1: I draw the scene in a texture using a framebuffer. After the ovrHmd_BeginFrame(hmd, 0) statement, I bind the framebuffer for offscreen rendering:

note that textures[] contains the grid and the star, mixed using the Fragment Shader from shaderProgram

glBindBuffer(GL_FRAMEBUFFER, frameBuffer);
glBindVertexArray(vertexArrayObject);
glEnable(GL_DEPTH_TEST);
glUseProgram(shaderProgram);
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, textures[0]);
glActiveTexture(GL_TEXTURE1);
glBindTexture(GL_TEXTURE_2D, textures[1]);

framebuffer uis created in that function, called after init_ovr() but before init_render_ovr():

int init_framebuffers(){

    // Framebuffers
    //-----------------------------------------------
    // In order to display, it has to be "complete" (at least 1 color/depth/stencil buffer attached, 1color attachement, same number of multisamples, attachement completes)
    glGenFramebuffers(1, &frameBuffer);
    glBindFramebuffer(GL_FRAMEBUFFER, frameBuffer);


    // ---- Texture "Color Buffer"
    glGenTextures(1, &renderedTex);
    glBindTexture(GL_TEXTURE_2D, renderedTex);

    glTexImage2D(renderedTex, 0, GL_RGB, renderTargetSize.w / 2, renderTargetSize.h, 0, GL_RGB, GL_UNSIGNED_BYTE, NULL);

    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);

    // Attaching the color buffer to the frame Buffer
    glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, renderedTex, 0);



    // ---- RenderBuffer
    // Render Buffer (to be able to render Depth calculation)
    GLuint rboDepthStencil;
    glGenRenderbuffers(1, &rboDepthStencil);
    glBindRenderbuffer(GL_RENDERBUFFER, rboDepthStencil);
    glRenderbufferStorage(GL_RENDERBUFFER, GL_DEPTH24_STENCIL8, renderTargetSize.w / 2, renderTargetSize.h);

    // Attaching the render buffer to the framebuffer
    glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_DEPTH_STENCIL_ATTACHMENT, GL_RENDERBUFFER, rboDepthStencil);


    // Binding the Frame Buffer so the rendering happens in it
    glBindFramebuffer( GL_FRAMEBUFFER, frameBuffer );

    return 0;
}

---

*Edit 2: Ok, so here's my rendering loop right now:

ovrFrameTiming hdmFrameTiming = ovrHmd_BeginFrame(hmd, 0);

        for (int eyeIndex = 0; eyeIndex < ovrEye_Count; eyeIndex++){

            ovrEyeType eye = hmdDesc.EyeRenderOrder[eyeIndex];
            ovrPosef eyePose = ovrHmd_BeginEyeRender(hmd, eye);

            // Clear the screen and the depth buffer (as it is filled with 0 initially, 
            // nothing will be draw (0 = on top);
            glClearColor(0.0f, 0.0f, 0.3f, 1.0f);
            glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_FRAMEBUFFER);

            // Drawing in the FrameBuffer
            glBindBuffer(GL_FRAMEBUFFER, frameBuffer);
            glBindVertexArray(vertexArrayObject);
            glEnable(GL_DEPTH_TEST);
            glUseProgram(shaderProgram);
            glActiveTexture(GL_TEXTURE0);
            glBindTexture(GL_TEXTURE_2D, textures[0]);
            glActiveTexture(GL_TEXTURE1);
            glBindTexture(GL_TEXTURE_2D, textures[1]);


            if (eye == ovrEye_Right){
                glScissor(renderTargetSize.w / 2, 0, renderTargetSize.w / 2, renderTargetSize.h);
                glViewport(renderTargetSize.w / 2, 0, renderTargetSize.w / 2, renderTargetSize.h);
            }else{
                glScissor(0, 0, renderTargetSize.w / 2, renderTargetSize.h);
                glViewport(0, 0, renderTargetSize.w / 2, renderTargetSize.h);
            }


            if (eye == ovrEye_Right)
                glClearColor(0.0f, 0.3f, 0.0f, 1.0f);
            else
                glClearColor(0.3f, 0.0f, 0.0f, 1.0f);
            glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);



            //Turn around Z
            trans = glm::rotate(
                trans,
                0.7f,
                glm::vec3(0.0f, 0.0f, 1.0f)
            );
            glUniformMatrix4fv(uniTrans, 1, GL_FALSE, glm::value_ptr(trans));

            // Drawing
            glDrawArrays(GL_TRIANGLES, 0, 36);

            // Unbind the custom frame Buffer
            glBindFramebuffer(GL_FRAMEBUFFER, 0);

            ovrHmd_EndEyeRender(hmd, eye, eyePose, &EyeTexture[eye].Texture);

        }

        ovrHmd_EndFrame(hmd);

And my render configuration:

EyeTexture[0].OGL.Header.API = ovrRenderAPI_OpenGL;
EyeTexture[0].OGL.Header.TextureSize = recommendedTex0Size;
EyeTexture[0].OGL.Header.RenderViewport.Size = recommendedTex0Size;
EyeTexture[0].OGL.Header.RenderViewport.Pos.x = 0;
EyeTexture[0].OGL.Header.RenderViewport.Pos.y = 0;
EyeTexture[0].OGL.TexId = renderedTex;


EyeTexture[1].OGL.Header.API = ovrRenderAPI_OpenGL;
EyeTexture[1].OGL.Header.TextureSize = recommendedTex1Size;
EyeTexture[1].OGL.Header.RenderViewport.Size = recommendedTex1Size;
EyeTexture[1].OGL.Header.RenderViewport.Pos.x = recommendedTex1Size.w;
EyeTexture[1].OGL.Header.RenderViewport.Pos.y = 0;
EyeTexture[1].OGL.TexId = renderedTex;

But I still can't manage to get anything displayed.

---

Original post:

So now I want to distort this texture into a barrel using the Oculus SDK. To do that, I initialize the Oculus engine:

int init_ovr(){

    // Init the OVR library
    ovr_Initialize();

    // Create the software device and connect the physical device
    hmd = ovrHmd_Create(0);
    if (hmd)
        ovrHmd_GetDesc( hmd, &hmdDesc );
    else
        return 1;

    //Configuring the Texture size (bigger than screen for barrel distortion)
    recommendedTex0Size = ovrHmd_GetFovTextureSize(hmd, ovrEye_Left, hmdDesc.DefaultEyeFov[0], 1.0f);
    recommendedTex1Size = ovrHmd_GetFovTextureSize(hmd, ovrEye_Right, hmdDesc.DefaultEyeFov[1], 1.0f);

    renderTargetSize.w = recommendedTex0Size.w + recommendedTex1Size.w;
    renderTargetSize.h = std::max( recommendedTex0Size.h, recommendedTex1Size.h );


    return 0;
}

and the rendering:

int init_render_ovr(){

    // Configure rendering with OpenGL
    ovrGLConfig cfg;
    cfg.OGL.Header.API = ovrRenderAPI_OpenGL;
    cfg.OGL.Header.RTSize = OVR::Sizei( hmdDesc.Resolution.w, hmdDesc.Resolution.h );
    cfg.OGL.Header.Multisample = 0;
    cfg.OGL.Window = sdl_window_info.info.win.window;

    ovrFovPort eyesFov[2];
    // I also tried =  { hmdDesc.DefaultEyeFov[0], hmdDesc.DefaultEyeFov[1] };

    if ( !ovrHmd_ConfigureRendering(hmd, &cfg.Config, ovrDistortionCap_Chromatic | ovrDistortionCap_TimeWarp, eyesFov, eyesRenderDesc) )
        return 1;

    EyeTexture[0].OGL.Header.API = ovrRenderAPI_OpenGL;
    EyeTexture[0].OGL.Header.TextureSize = recommendedTex0Size;
    EyeTexture[0].OGL.Header.RenderViewport = eyesRenderDesc[0].DistortedViewport;
    EyeTexture[0].OGL.TexId = renderedTex;

    EyeTexture[1].OGL.Header.API = ovrRenderAPI_OpenGL;
    EyeTexture[1].OGL.Header.TextureSize = recommendedTex1Size;
    EyeTexture[1].OGL.Header.RenderViewport = eyesRenderDesc[1].DistortedViewport;
    EyeTexture[1].OGL.TexId = renderedTex;

    return 0;
}

Finally, I enter my main rendering loop

        ovrFrameTiming hdmFrameTiming = ovrHmd_BeginFrame(hmd, 0);

        for (int eyeIndex = 0; eyeIndex < ovrEye_Count; eyeIndex++){

            ovrEyeType eye = hmdDesc.EyeRenderOrder[eyeIndex];
            ovrPosef eyePose = ovrHmd_BeginEyeRender(hmd, eye);

            if (eye == ovrEye_Right){
                glScissor(renderTargetSize.w / 2, 0, renderTargetSize.w / 2, renderTargetSize.h);
                glViewport(renderTargetSize.w / 2, 0, renderTargetSize.w / 2, renderTargetSize.h);
            }else{
                glScissor(0, 0, renderTargetSize.w / 2, renderTargetSize.h);
                glViewport(0, 0, renderTargetSize.w / 2, renderTargetSize.h);
            }


            if (eye == ovrEye_Right)
                glClearColor(0.0f, 0.3f, 0.0f, 1.0f);
            else
                glClearColor(0.3f, 0.0f, 0.0f, 1.0f);
            glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);



            //Turn around Z
            trans = glm::rotate(
                trans,
                0.7f,
                glm::vec3(0.0f, 0.0f, 1.0f)
            );
            glUniformMatrix4fv(uniTrans, 1, GL_FALSE, glm::value_ptr(trans));

            // Drawing
            glDrawArrays(GL_TRIANGLES, 0, 36);


            ovrHmd_EndEyeRender(hmd, eye, eyePose, &EyeTexture[eye].Texture);

        }
        ovrHmd_EndFrame(hmd);

But the result is a plain, black screen. I tried to move the glViewport all over, to manually set the EyeRenderDesc structure, I read twice the documentation and followed it quite strictly... But nothing helps.

Did I forgot something, somewhere? I can't get any ideas where to look on right now.

Answer 1

Several issues with the code. First off, you're setting up the EyeTexture structures incorrectly.

EyeTexture[0].OGL.Header.TextureSize = recommendedTex0Size;
EyeTexture[0].OGL.Header.RenderViewport = eyesRenderDesc[0].DistortedViewport;

The DistortedViewport value you're using here specifies the region on the physical screen where the distorted view of the rendered scene will be placed. The texture RenderViewport value on the other hand is supposed to be the region of the offscreen texture to which you've rendered the scene. Since you almost always want to render to the full texture, it's usually sufficient to do this:

EyeTexture[0].OGL.Header.TextureSize = recommendedTex0Size;
EyeTexture[0].OGL.Header.RenderViewport.Size = recommendedTex0Size;
EyeTexture[0].OGL.Header.RenderViewport.Pos.x = 0;
EyeTexture[0].OGL.Header.RenderViewport.Pos.y = 0;

You're setting the same texture ID in both locations.

EyeTexture[0].OGL.TexId = renderedTex;
EyeTexture[1].OGL.TexId = renderedTex;

You can do this, but then you have to make the texture twice as wide and set up the respective RenderViewport values to each target one half of the texture. Personally I just use a distinct texture for each eye to avoid that.

Next, in the code sample you've provided you're not rendering to an offscreen buffer.

for (int eyeIndex = 0; eyeIndex < ovrEye_Count; eyeIndex++){
   ovrEyeType eye = hmdDesc.EyeRenderOrder[eyeIndex];
   ovrPosef eyePose = ovrHmd_BeginEyeRender(hmd, eye);

   ... lots of openGL viewport and draw calls ...

   ovrHmd_EndEyeRender(hmd, eye, eyePose, &EyeTexture[eye].Texture);
}

However, for the Oculus SDK distortion to work, you can't be drawing directly to the primary framebuffer. You need to create offscreen framebuffers targeting the textures identified in the EyeTexture array.

for (int eyeIndex = 0; eyeIndex < ovrEye_Count; eyeIndex++){
   ovrEyeType eye = hmdDesc.EyeRenderOrder[eyeIndex];
   ovrPosef eyePose = ovrHmd_BeginEyeRender(hmd, eye);

   ... activate offscreen framebuffer ...

   ... lots of openGL viewport and draw calls ...

   ... decativate offscreen framebuffer ...

   ovrHmd_EndEyeRender(hmd, eye, eyePose, &EyeTexture[eye].Texture);
}

You can see a full example of this here. Of course that example has been updated for 0.4.x, so you might want to go back in the history until you see ovrHmd_EndEyeRender (it's been dropped in the latest SDK).

EDIT:

I took a look at your posted code. It's incomplete because you didn't include the headers you're relying on. Regardless, there are still a few obvious issues:

            // Clear the screen and the depth buffer (as it is filled with 0 initially,
            // nothing will be draw (0 = on top);
            glClearColor(0.0f, 0.0f, 0.3f, 1.0f);
            glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_FRAMEBUFFER);

            // Drawing in the FrameBuffer
            glBindBuffer(GL_FRAMEBUFFER, frameBuffer);
            glBindRenderbuffer(GL_RENDERBUFFER, rboDepthStencil);
            glEnable(GL_DEPTH_TEST);

• GL_FRAMEBUFFER is not a valid parameter for glClear. I have no idea what effect it's going to have, but it's probably not going to be whatever you expect.
• Calling glClear before calling glBindBuffer means you're not actually clearing the offscreen texture target. You're clearing the actual draw framebuffer, which the SDK will do anyway before performing distortion. Move your glClear call to inside the framebuffer operations.
• You don't need to call glBindRenderbuffer. I'm not sure what effect that's going to have, but if you've already attached the depthStencil renderbuffer to the framebuffer object then it's going to be pulled in and used automatically when you activate the framebuffer.
• You create the renderedTex texture, bind it, set some values, and then just leave it bound. It should only be bound long enough to set the parameters. It should not be bound when you call glFramebufferTexture2D, nor in fact should it be bound by you at any time after you set it up.
• in init_render_ovr you fully set up EyeTexture[0], but you only set up a couple of values in EyeTexture1. The EyeTexture1 values don't default to those in EyeTexture[0]. You have to initialize every member.

• Until you've solved your black screen problem, your code should be liberally peppered with glGetError() calls. I have a GL_CHECK_ERROR which on debug builds expands to something like this:

  GLenum errorCode = glGetError(); if (errorCode != 0) { throw error(errorCode); }