Thursday, 28 January 2021

Poor performance when continuously updating a SurfaceView with pixel data from a ByteBuffer

I am writing an Android app in which a native thread keeps generating image frames (as raw pixel data) at about 60 fps, which are then supposed to be displayed in a SurfaceView. Currently, the thread writes pixel data to a direct ByteBuffer shared between the native thread and the JVM, then invokes a callback via JNI to notify the JVM side that a frame is ready; the buffer is then read into a Bitmap and drawn on the SurfaceView. My code looks roughly like this (full source code would be too big to share):

// this is a call into native code that retrieves
// the width and height of the frame in pixels
// returns something on the order of 320×200
private external fun getFrameDimensions(): (Int, Int)

// a direct ByteBuffer shared between the JVM and the native thread
private val frameBuffer: ByteBuffer

private fun getFrame(): Bitmap {
    val (width, height) = getFrameDimensions()

    return Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888).apply {
        setHasAlpha(false)
        copyPixelsFromBuffer(frameBuffer.apply {
            order(ByteOrder.nativeOrder())
            rewind()
        })
    }
}

// the SurfaceView widget which should display the image
private lateinit var surfaceView: SurfaceView

private val SCALE = 8

// callback invoked by the native thread
public fun onFrameReady() {
    val canvas: Canvas = surfaceView.holder.lockCanvas() ?: return
    val bitmap = getFrame()
    
    try {
        canvas.drawBitmap(bitmap.scale(
            bitmap.width * SCALE,
            bitmap.height * SCALE),
            0.0f, 0.0f, null
        )
    } finally {
        holder.unlockCanvasAndPost(canvas)
    }
}

The good news is that the above works: the screen updates at roughly the expected frame rate. However, performance is really poor, to the point where the app locks up: it stops responing to e.g. pressing the button and eventually an ANR message pops up. Clearly I am doing something wrong, but I am not quite able to tell what exactly.

Is there a way to make the above run faster? Preferably I would like to avoid writing more native code than necessary. I would especially like to avoid putting anything Android-specific at the native side of things.



from Poor performance when continuously updating a SurfaceView with pixel data from a ByteBuffer

No comments:

Post a Comment