Sunday, 20 January 2019

How to display a Texture in Unity edited by an Android plugin

I can't get the texture from a SurfaceTexture to display in anything but a TextureView.

I generate a Texture in Java and pass its pointer back to Unity:

public void initGLTexture()
{
    Log.d("Unity", "initGLTexture");
    int textures[] = new int[1];
    GLES20.glGenTextures(1, textures, 0);
    mTextureId = textures[0];

    GLES20.glBindTexture(GL_TEXTURE_EXTERNAL_OES, mTextureId);
    GLES20.glTexParameterf(GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR);
    GLES20.glTexParameterf(GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);
    GLES20.glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
    GLES20.glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
}

I create a SurfaceTexture from the id (in Java):

mSurfaceTexture = new SurfaceTexture(mTextureId);

I use a third-party library, GeckoView, to render onto the Surface of the SurfaceTexture. I call the following method from Unity's OnRenderObject() to keep all GL rendering on the same thread:

mSurfaceTexture.updateTexImage();

I know the above code allows proper drawing onto the surface as well as conversion to a texture because I can get images of the texture AS LONG AS I instantiate a TextureView, see this. Why is the TextureView necessary to get proper data from the SurfaceTexture?

I call the following in Unity to load the texture:

_imageTexture2D = Texture2D.CreateExternalTexture(
        600,600,TextureFormat.RGBA32,false,true,(IntPtr) mTextureId);
_rawImage.texture = _imageTexture2D;

Why does the RawImage with the texture applied show only this sprite-looking thing, which should be a webpage?

texture

I notice that the SurfaceTexture's onFrameAvailable is not called when it has a consumer it recognizes, i.e. the textureview, but it is called when its just the RawImage consuming its frames.

UPDATE: So I've been working on the hypothesis of: use Gecko to draw to the Surface, and use a SurfaceTexture to render this surface to a GL_TEXTURE_EXTERNAL_OES. Since I can't display this on Unity (not sure why) I am drawing this texture to a frame buffer and copying the pixels in the framebuffer to a GL_TEXTURE_2D. I am getting some semblance of a web page in the texture_2d (in the emulator with an imageview and glReadPixels) but I think my shaders are off. However, when I import the work into Unity to test if the pipeline is okay thus far I just get a black screen (not even the distorted stuff I got in the emulator).



from How to display a Texture in Unity edited by an Android plugin

No comments:

Post a Comment