I'm using a Texture widget, rendering its content from native code using OpenGL ES. In native code I call ANativeWindow_fromSurface
and from that create an EGL surface. AIUI what happens is:
- The
ANativeWindow
represents the producer side of a buffer queue. - Calling
eglSwapBuffers
causes a texture to be sent to this queue. - Flutter receives the texture and renders it using Skia when the
TextureLayer
is painted. - The texture is scaled to match the size of the
TextureLayer
(the scaling happens inAndroidExternalTextureGL::Paint()
).
I'm trying to figure out how to synchronise the OpenGL rendering. I think I can use the choreographer to synchronise with the display vsync, but I'm unclear on how much latency this bufferqueue-then-render-with-skia mechanism introduces. I don't see any means to explicitly synchronise my native code's generation of textures with the TextureLayer
's painting of them.
The scaling appears to be a particularly tricky aspect. I would like to avoid it entirely, by ensuring that the textures the native code generates are always of the right size. However there doesn't appear to be any direct link between the size of the TextureLayer
and the size of the Surface/ANativeWindow. I could use a SizeChangedLayoutNotifier (or one of various alternative hacks) to detect changes in the size and communicate them to the native code, but I think this would lag by at least a frame so scaling would still take place when resizing.
I did find this issue, which talks about similar resizing challenges, but in the context of using an OEM web view. I don't understand Hixie's detailed proposal in that issue, but it appears to be specific to embedding of OEM views so I don't think it would help with my case.
Perhaps using a Texture widget here is the wrong approach. It seems to be designed mainly for displaying things like videos and camera previews. Is there another way to host natively rendered, interactive OpenGL graphics in Flutter?
from How to synchronize native OpenGL rendering when using flutter's Texture widget on Android
No comments:
Post a Comment