Sunday, 20 December 2020

JS UI is blocked while processing image analysis of cameraX

I have integrated cameraX through bridging and can able to get the preview on JS. But the problem, JS UI is getting blocked when I start analyzing the image and convert to bitmap.

From analyze part, I get 16 fps (even actual fps is low from cameraX) in second before convert into bitmap. But when I convert, its drastically dropping down to 6-7 fps.

private ImageAnalysis setImageAnalysis() {
    cameraExecutor = Executors.newSingleThreadExecutor();
    ImageAnalysis imageAnalysis =
            new ImageAnalysis.Builder()
                    .setTargetResolution(new Size(640, 480))
                    .setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
                    .setImageQueueDepth(1)
                    .build();

    imageAnalysis.setAnalyzer(cameraExecutor, new ImageAnalysis.Analyzer() {

        @Override
        public void analyze(@NonNull ImageProxy image) {

            / * is UI lagging due to heavy work on each frame ?? */
            //from frame to bitmap
            Image image_ = image.getImage();
            final Bitmap bitmap = toBitmap(image_); 

            //final Bitmap bitmap = mPreviewView.getBitmap();

            if(bitmap != null){
                Mat rgba = new Mat();
                Utils.bitmapToMat(bitmap, rgba); // using opencv lib

                if (callback != null) {
                    // this runs on the handler thread
                    HOCalc.getInstance().process(rgba, new HOCalc.ResultsCallback() {
                        @Override
                        public void onResults(...) {
                            if (getContext() == null) return;
                            callback.onResults(...);
                        }
                    }, new WeakReference<>(getContext()));
                }
            }
            image.close();
        }
    });
    return imageAnalysis;
}

HOCalc:

public class HOCalc {
    private HOCalc() {
        HandlerThread handlerThread = new HandlerThread("HandlerThread");
        handlerThread.start();
        Looper looper = handlerThread.getLooper();
        mHandler = new Handler(looper) {
            @Override
            public void handleMessage(Message msg) {
                ...
                 execute(..., ...);
            }
        };
    }
    
   static HOCalc getInstance() {
        if (instance == null) {
            instance = new HOCalc();
        }
        return instance;
    }
   
    private void executeRecognition(...) {
      Context ctx = context.get();
      if (ctx == null) {
        finishExecution(...);
        return;
      }

      Handler handler = new Handler(ctx.getMainLooper());
      try {
        handler.post(new Runnable() {
            @Override
            public void run() {
                    callback.onResults(...);
                if (isProcessing != null) isProcessing.set(false);
            }
        });
      } catch (JsonSyntaxException e) {
        finishExecution(...);
      }
    }
   
    void process(Mat m, final ResultsCallback callback, final WeakReference<Context> context) {
        if (!mHandler.hasMessages(10) && isProcessing.compareAndSet(false, true)) 
        {
            ...
            Message mes = mHandler.obtainMessage(10, 0, 0, map);
            mHandler.sendMessage(mes);
        }
    }
}

I need to unblock JS UI part which is running a few animations. RN UI is not 60fps.

How can I make it work better performance without affecting the JS UI Animation?



from JS UI is blocked while processing image analysis of cameraX

No comments:

Post a Comment