Android glTexSubImage2D with GL_TEXTURE_EXTERNAL_OES problem (OpenGL + OpenCV + CameraX)

Question

I am writing an app utilizing CameraX to capture the camera feed, OpenCV to modify the frames and the OpenGL to render them on a GLSurfaceView.

My current approach was to render the original frames from CameraX, construct a pixel buffer, an OpenCV Mat, use glReadPixels to copy the pixel buffer (ByteBuffer) into the Mat, then process it, write the pixel buffer from Mat back to the pixel buffer and then render it onto the texture with glTexSubImage2D.

The problem is, CameraX enables me to access the frame data via GL_TEXTURE_EXTERNAL_OES, which - as per OpenGL's documentation - does not allow for glTexSubImage2D to modify the texture and my current approach, using the code below, does not work and continously throws W/OpenGLRenderer: [SurfaceTexture-1-25873-0] bindTextureImage: clearing GL error: 0x500, which is expected as says the documentation. Is there an alternative way to copy the buffer into GL_TEXTURE_EXTERNAL_OES after OpenCV processing? Another thing I can think of is to copy the texture into GL_TEXTURE_2D and then use glTexSubImage2D, although I have no idea how to do that. Maybe someone knows another approach that would allow me to achieve the goal?

My code:

public void onDrawFrame(GL10 gl) {
    GLES20.glClear( GLES20.GL_COLOR_BUFFER_BIT );

    if(this.resolutionSet) {
        surfaceTexture.updateTexImage();

        GLES20.glUseProgram(hProgram);

        int ph = GLES20.glGetAttribLocation(hProgram, "vPosition");
        int tch = GLES20.glGetAttribLocation(hProgram, "vTexCoord");
        int th = GLES20.glGetUniformLocation(hProgram, "sTexture");

        GLES20.glActiveTexture(GLES20.GL_TEXTURE0);
        GLES20.glBindTexture(GL_TEXTURE_EXTERNAL_OES, hTex[0]);
        GLES20.glUniform1i(th, 0);

        GLES20.glVertexAttribPointer(ph, 2, GLES20.GL_FLOAT, false, 4 * 2, pVertex);
        GLES20.glVertexAttribPointer(tch, 2, GLES20.GL_FLOAT, false, 4 * 2, pTexCoord);
        GLES20.glEnableVertexAttribArray(ph);
        GLES20.glEnableVertexAttribArray(tch);

        GLES20.glReadPixels(0, 0, this.img.cols(), this.img.rows(), GL_RGBA, GL_UNSIGNED_BYTE, this.imgBuff);

        img.put(0, 0, this.imgBuff.array());

        Log.d(LOG_TAG, img.toString());

        Imgproc.cvtColor(img, img, Imgproc.COLOR_RGBA2BGRA);
        Imgproc.cvtColor(img, img, Imgproc.COLOR_BGRA2GRAY);

        this.imgBuff.clear();
        img.get(0, 0, this.imgBuff.array());

        GLES20.glTexImage2D(GL_TEXTURE_EXTERNAL_OES, 0, GL_RGBA, img.width(), img.height(), 0, GL_RGBA, GL_UNSIGNED_BYTE, null);
        GLES20.glTexSubImage2D(GL_TEXTURE_EXTERNAL_OES, 0, 0, 0, img.width(), img.height(), GL_RGBA, GL_UNSIGNED_BYTE, this.imgBuff);
        GLES20.glDrawArrays(GLES20.GL_TRIANGLE_STRIP, 0, 4);
    }

    GLES20.glFlush();
}

and the code initializing texture:

    hTex = new int[1];
    GLES20.glGenTextures ( 1, hTex, 0 );
    GLES20.glBindTexture(GL_TEXTURE_EXTERNAL_OES, hTex[0]);
    GLES20.glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE);
    GLES20.glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE);
    GLES20.glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_NEAREST);
    GLES20.glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_NEAREST);

solidpixel solidpixel · Accepted Answer · 2020-02-02T16:17:11

Given you are replacing the entire image and uploading "normal" RGBA data from the application, why does this need to be a GL_TEXTURE_EXTERNAL_OES type at all? Just use a normal GL_TEXTURE_2D.

Android glTexSubImage2D with GL_TEXTURE_EXTERNAL_OES problem (OpenGL + OpenCV + CameraX)

1 Answers