I am developing a camera application that users Camera API 2. I have managed to get the image capturing and video recording functions to works. But whenever I perform either of those things my camera preview freezes. I am trying to understand what changes I need to make in order to work it fine without any issues.
After handling camera permission and selecting the camera id I invoke the following function to get a camera preview. this works fine without any issues.
private fun startCameraPreview(){
if (ContextCompat.checkSelfPermission(requireContext(), Manifest.permission.CAMERA)
== PackageManager.PERMISSION_GRANTED &&
ContextCompat.checkSelfPermission(requireContext(), Manifest.permission.RECORD_AUDIO)
== PackageManager.PERMISSION_GRANTED){
lifecycleScope.launch(Dispatchers.Main) {
camera = openCamera(cameraManager, cameraId!!, cameraHandler)
val cameraOutputTargets = listOf(viewBinding.cameraSurfaceView.holder.surface)
session = createCaptureSession(camera, cameraOutputTargets, cameraHandler)
val captureBuilder = camera.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW)
captureBuilder.set(
CaptureRequest.CONTROL_AF_MODE,
CaptureRequest.CONTROL_AF_MODE_CONTINUOUS_PICTURE)
captureBuilder.addTarget(viewBinding.cameraSurfaceView.holder.surface)
session.setRepeatingRequest(captureBuilder.build(), null, cameraHandler)
selectMostMatchingImageCaptureSize()
selectMostMatchingVideoRecordSize()
}
}else{
requestCameraPermission()
}
}
According to my understanding when you create CaptureSession with relevant targets and when you make setRepeatingRequest out of it we get the camera preview. correct me if I'm wrong.
Here is the function I use to capture an image.
private fun captureImage(){
if (captureSize != null) {
captureImageReader = ImageReader.newInstance(
captureSize!!.width, captureSize!!.height, ImageFormat.JPEG, IMAGE_BUFFER_SIZE)
viewModel.documentPreviewSizeToCaptureSizeScaleFactor =
captureSize!!.width / previewSize!!.width.toFloat()
lifecycleScope.launch(Dispatchers.IO) {
val cameraOutputTargets = listOf(
viewBinding.cameraSurfaceView.holder.surface,
captureImageReader.surface
)
session = createCaptureSession(camera, cameraOutputTargets, cameraHandler)
takePhoto().use { result ->
Log.d(TAG, "Result received: $result")
// Save the result to disk
val output = saveResult(result)
Log.d(TAG, "Image saved: ${output.absolutePath}")
// If the result is a JPEG file, update EXIF metadata with orientation info
if (output.extension == "jpg") {
decodedExifOrientationOfTheImage =
decodeExifOrientation(result.orientation)
val exif = ExifInterface(output.absolutePath)
exif.setAttribute(
ExifInterface.TAG_ORIENTATION, result.orientation.toString()
)
exif.saveAttributes()
Log.d(TAG, "EXIF metadata saved: ${output.absolutePath}")
}
}
}
}
}
The function takePhoto() is a function I have placed in the inherited base fragment class which is responsible for setting up capture requests and saving the image.
protected suspend fun takePhoto(): CombinedCaptureResult = suspendCoroutine { cont ->
// Flush any images left in the image reader
@Suppress("ControlFlowWithEmptyBody")
while (captureImageReader.acquireNextImage() != null) {
}
// Start a new image queue
val imageQueue = ArrayBlockingQueue<Image>(IMAGE_BUFFER_SIZE)
captureImageReader.setOnImageAvailableListener({ reader ->
val image = reader.acquireNextImage()
Log.d(TAG, "Image available in queue: ${image.timestamp}")
imageQueue.add(image)
}, imageReaderHandler)
val captureRequest = session.device.createCaptureRequest(
CameraDevice.TEMPLATE_STILL_CAPTURE
).apply {
addTarget(captureImageReader.surface)
}
session.capture(captureRequest.build(), object : CameraCaptureSession.CaptureCallback() {
override fun onCaptureCompleted(
session: CameraCaptureSession,
request: CaptureRequest,
result: TotalCaptureResult
) {
super.onCaptureCompleted(session, request, result)
val resultTimestamp = result.get(CaptureResult.SENSOR_TIMESTAMP)
Log.d(TAG, "Capture result received: $resultTimestamp")
// Set a timeout in case image captured is dropped from the pipeline
val exc = TimeoutException("Image dequeuing took too long")
val timeoutRunnable = Runnable { cont.resumeWithException(exc) }
imageReaderHandler.postDelayed(timeoutRunnable, IMAGE_CAPTURE_TIMEOUT_MILLIS)
// Loop in the coroutine's context until an image with matching timestamp comes
// We need to launch the coroutine context again because the callback is done in
// the handler provided to the `capture` method, not in our coroutine context
@Suppress("BlockingMethodInNonBlockingContext")
lifecycleScope.launch(cont.context) {
while (true) {
val image = imageQueue.take()
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.Q &&
image.format != ImageFormat.DEPTH_JPEG &&
image.timestamp != resultTimestamp
) continue
Log.d(TAG, "Matching image dequeued: ${image.timestamp}")
// Unset the image reader listener
imageReaderHandler.removeCallbacks(timeoutRunnable)
captureImageReader.setOnImageAvailableListener(null, null)
// Clear the queue of images, if there are left
while (imageQueue.size > 0) {
imageQueue.take().close()
}
// Compute EXIF orientation metadata
val rotation = relativeOrientation.value ?: defaultOrientation()
Log.d(TAG,"EXIF rotation value $rotation")
val mirrored = characteristics.get(CameraCharacteristics.LENS_FACING) ==
CameraCharacteristics.LENS_FACING_FRONT
val exifOrientation = computeExifOrientation(rotation, mirrored)
// Build the result and resume progress
cont.resume(
CombinedCaptureResult(
image, result, exifOrientation, captureImageReader.imageFormat
)
)
}
}
}
}, cameraHandler)
}
Invoking these functions above does perform image capturing but it freezes the preview. If I want to get the preview back I need to reset the preview using the bellow function. I have to call this method end of captureImage() function.
private fun resetCameraPreview(){
lifecycleScope.launch(Dispatchers.Main) {
val cameraOutputTargets = listOf(viewBinding.cameraSurfaceView.holder.surface)
session = createCaptureSession(camera, cameraOutputTargets, cameraHandler)
val captureBuilder = camera.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW)
captureBuilder.set(
CaptureRequest.CONTROL_AF_MODE,
CaptureRequest.CONTROL_AF_MODE_CONTINUOUS_PICTURE)
captureBuilder.addTarget(viewBinding.cameraSurfaceView.holder.surface)
session.setRepeatingRequest(captureBuilder.build(), null, cameraHandler)
}
}
Even doing this is not providing a good User experience as it freezes the preview for few seconds. When it's come to video recording this issue becomes unsolvable using even about not so good fix.
This is the function I use for video recording.
private fun startRecoding(){
if (videoRecordeSize != null) {
lifecycleScope.launch(Dispatchers.IO) {
configureMediaRecorder(videoRecodeFPS,videoRecordeSize!!)
val cameraOutputTargets = listOf(
viewBinding.cameraSurfaceView.holder.surface,
mediaRecorder.surface
)
session = createCaptureSession(camera, cameraOutputTargets, cameraHandler)
recordVideo()
}
}
}
The function recordVideo() is a function in the inherited base fragment class which is responsible for setting up capture requests and starting mediaRecorder to begin video recode to a file.
protected fun recordVideo() {
lifecycleScope.launch(Dispatchers.IO) {
val recordRequest = session.device
.createCaptureRequest(CameraDevice.TEMPLATE_RECORD).apply {
addTarget(mediaRecorder.surface)
}
session.setRepeatingRequest(recordRequest.build(), null, cameraHandler)
mediaRecorder.apply {
start()
}
}
}
While it does record the video correctly and saves the file when invoked mediaRecorder.stop(). But the whole time the camera preview is freeze even after calling mediaRecorder.stop().
What am I missing here? both of the times when I create capture sessions I have included preview surface as a target. Doesn't it enough for Camera 2 API to know that it should push frames to the surface while capturing images or recording videos? You can find the repo for this codebase here. I hope someone can help me because I'm stuck with Camera 2 API. I wish I could use cameraX but some parts are still in beta so I can't use it in production.
from Camera 2 custom implementation preview freeze when preform image capture or video recode
No comments:
Post a Comment