Tuesday, 18 April 2023

Object Detection - draw rectangle around objects using bounding box

First of all, I want to address that I know a lot of similar questions have been posted here on SO, but none of them provide an answer of explaining steps and logic behind it - just simply points to different tutorial repo's.

I've went through few of the demo repo's that show how this can be done. Majority of the solutions differ, but sadly they include 5 classes with very complicated code for such an easy task, so I want to look for:

  1. Solution that does what is needed - draw rectangle around detected objects (multiple) while live feed is running
  2. Code should be simplistic, elegant and easy to understand
  3. I'd love to understand the logic behind it as well (which is not possible from demo repos)

So here's what I've.

The big challenge is to transform the detected bounding box coordinates so they make sense when you display them on your Preview. Preview image has different size than input image and thus, coordinates should be adjusted accordingly.

For now, I've a problem that draws rectangles with a huge error (rectangles are off from actual object, so coordinates are incorrectly calculated).

override fun draw(canvas: Canvas) {
    super.draw(canvas)

    // Get the dimensions of the preview image
    val previewWidth = previewImageSize.width
    val previewHeight = previewImageSize.height

    // Get the dimensions of the input image
    val inputWidth = inputImageSize.width
    val inputHeight = inputImageSize.height

    // Calculate the scaling factor for each dimension
    val scaleX: Float = previewWidth.toFloat() / inputWidth.toFloat()
    val scaleY: Float = previewHeight.toFloat() / inputHeight.toFloat()

    for (result in results) {
        val boundingBox = result.boundingBox

        val left = boundingBox.left * scaleX
        val top = boundingBox.top * scaleY
        val right = boundingBox.right * scaleX
        val bottom = boundingBox.bottom * scaleY

        // Draw bounding box around detected objects
        val drawableRect = RectF(left, top, right, bottom)
        canvas.drawRect(drawableRect, boxPaint)

        // Create text to display alongside detected objects
        val drawableText =
            result.categories[0].label + " " +
                    String.format("%.2f", result.categories[0].score)

        // Draw rect behind display text
        textBackgroundPaint.getTextBounds(drawableText, 0, drawableText.length, bounds)
        val textWidth = bounds.width()
        val textHeight = bounds.height()
        canvas.drawRect(
            left,
            top,
            left + textWidth + BOUNDING_RECT_TEXT_PADDING,
            top + textHeight + BOUNDING_RECT_TEXT_PADDING,
            textBackgroundPaint
        )

        // Draw text for detected object
        canvas.drawText(drawableText, left, top + bounds.height(), textPaint)
    }
}

Additional information:

Input image size: 320x320 (resized image from preview that is passed inside the model to get the result)

Preview size: 1857x1080 (1857 as a height)

ImageAnalsys image that is received from CameraX: W:640, H:480

Calc. ScaledX value: 3.375

Calc. ScaledY value: 5.803125

Current result:

enter image description here



from Object Detection - draw rectangle around objects using bounding box

No comments:

Post a Comment