Thursday, 9 February 2023

Allow for Multiple-Pass Renders in GLSurfaceView Renderer with SurfaceTexture

I am displaying a video on a GLSurfaceView with a custom renderer that requires that multiple shaders be applied in succession. Currently, it is working successfully with one shader, though I am not sure how to extend the rendering pipeline to apply multiple shaders in succession.

I know that there are some examples concerning applying multiple shaders (using FrameBuffers and RenderBuffers), but I have not found any that deal with an image passed in through a SurfaceTexture.

There is a specific concern I would like to address:

A SurfaceTexture must be bound to a GL_TEXTURE_EXTERNAL_OES texture. On the other hand, a FrameBuffer cannot be bound to a GL_TEXTURE_EXTERNAL_OES texture (typically a GL_TEXTURE_2D is used), so is it even possible to use a FrameBuffer for a multi-pass render when the input texture is of a different format than the output? If not, what are the other options for performing a multi-pass render?

Below is some relevant code in the onSurfaceCreated function of the renderer I am trying to extend to perform multiple passes::

            GLES20.glGenTextures(1, this.textureID, 0);
            GLES20.glBindTexture(GL_TEXTURE_EXTERNAL_OES, this.textureID[0]);
            GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR);
            GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);
            GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE);
            GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE);
            GLES20.glBindTexture(GL_TEXTURE_EXTERNAL_OES, 0);
            this.surfaceTexture = new SurfaceTexture(this.textureID[0]);

Below is some relevant code in the onDrawFrame function of that renderer:

        synchronized (this) {
            if (this.updateSurface) {
                this.surfaceTexture.updateTexImage();
                this.surfaceTexture.getTransformMatrix(this.stMatrix);
                this.updateSurface = false;
            }
        }

        GLES20.glBindTexture(GL_TEXTURE_EXTERNAL_OES, this.textureID[0]);
        GLES20.glClearColor(0.5f, 0.5f, 0.5f, 1.0f);
        GLES20.glClear(GLES20.GL_DEPTH_BUFFER_BIT | GLES20.GL_COLOR_BUFFER_BIT);

  //apply shader here and call glDrawArrays() at end


from Allow for Multiple-Pass Renders in GLSurfaceView Renderer with SurfaceTexture

No comments:

Post a Comment