Strange bugs while rendering with OpenGL under Qml Scene

Question

As a hobby, I am experimenting with OpenGL and write my own silly engine with Qt+OpenGL. Last success was building Entity System Framework which could render scene with shadows (by render-to-texture to RenderBuffer) with attached physics engine (BulletPhysics).

After that I updated to Qt 5.2 and was totally expressed by Qml (not trying to use it before). I'm experimented about 2d qml graphics a bit, and decide to use qml for user interface, but reject 2d project and return to 3d. I created new qml project, try to render some stuff under qml as in Scene Graph - OpenGL Under QML example. After that I moved my old code from 3d project to new project, with some small refactoring (which should not affect anything).

In this moment I get tired with some strange bug. It seems like first frame is rendered as needed, but after that something goes wrong.

This is a some less or more normally frame (before any moving)

In the left top corner I am output depth buffer (shadow map). In the right top corner I make some custom scrollers with qml. I am using this scrollers for changing shadow map rotating and moving camera. When scrollers are centered, nothing moves. When I change something, scene became ugly and I can't wonder why.

This is the frame after some shadow map rotating

I see that shadow map depth buffer not changed (why? o_0) and shadow in scene goes wrong. But not only shadow is failed after refactoring my old code to qml project. When I move camera, bugs became more strange.

This is the frame after moving camera down (-Z)

Scene is rendered under qml by using:

connect(window, &QQuickWindow::beforeRendering, this, &MyApp::renderScene, Qt::DirectConnection);

In renderScene I:

• Do some OpenGL conveyor setup (to prevent changes by Qml Scene Graph, may be excess);

  glViewport(0, 0, window->width(), window->height());
  
  glClearColor(0, 0, 0, 1);
  glClearDepth(1);
  glDepthRange(0,1);
  glDepthFunc(GL_LEQUAL);
  

• Call BulletPhysics to calculate physics (in test scene - two falling cubes)

• Render shadow map (depth-only) to RenderBuffer (2048x2048 texture, GL_DEPTH_COMPONENT24)
• Render scene to RenderBuffer (two GL_RGBA8 textures for color and normals and one GL_DEPTH_COMPONENT24 for depth)
• Render two 2d quads - fullscreen (scene render with all textures from renderbuffer to render with shadows) and side screen (top left corner's texture from depthbuffer)

I am tired to hunt what is going on. glGetError shows nothing.

Moved camera one more time by XY-axes. Man's horn silhouette is still visible and make strange effect

Some more screenshots:

Old project (without qml)

Old project (big screenshot)

On new project, window resizing is broken too

What can it be?

Environment: Qt Creator 3.0.0, Qt 5.2.0, Linux (Kubuntu 13.10) 64-bit, HD 7750

OpenGL: in old project I set up 3.3, new project use 4.3 (as far as I remember)

P.S. Depth map is distorted because I use some additional distortion in shaders to improve shadows in scene center.

Answer 1

An inspiration came upon me after a day without an answer and many previous days!

I have suspected that something was wrong with the depth buffer and tried to output to debug texture ids that used to render to and used to render to fullscreen quad. I tried switch off render-to-texture buffer swapping and much more. But I can't even think that real problem is at Qml's side.

glDepthMask(GL_TRUE); // 42!

One this line is like 42 - answer to all questions. It's seems that Qml is calling glDepthMask(GL_FALSE); before first render frame (but after emit &QQuickWindow::beforeRendering signal). Result of this call was that only first my frame really rendered with writing to depth buffer. Why I think that glDepthMask(GL_FALSE); is called only once? Because FPS is noticeably decreased after I inserted my "42", and increased after I inserted glDepthMask(GL_FALSE); in the end of slot that used to render my scene.

Now I can freely rotate "Sun" (shadow) and move camera without any visual bugs.

Yippee!