Wednesday, 12 May 2021

Using opencv to find the most similar image that contains another image

If the title isn't clear let's say I have a list of images (10k+), and I have a target image I am searching for.

Here's an example of the target image:

enter image description here

Here's an example of images I will want to be searching to find something 'similar' (ex1, ex2, and ex3):

enter image description here

enter image description here

enter image description here

Here's the matching I do (I use KAZE)

from matplotlib import pyplot as plt
import numpy as np
import cv2
from typing import List
import os
import imutils


def calculate_matches(des1: List[cv2.KeyPoint], des2: List[cv2.KeyPoint]):
    """
    does a matching algorithm to match if keypoints 1 and 2 are similar
    @param des1: a numpy array of floats that are the descriptors of the keypoints
    @param des2: a numpy array of floats that are the descriptors of the keypoints
    @return:
    """
    # bf matcher with default params
    bf = cv2.BFMatcher(cv2.NORM_L2)
    matches = bf.knnMatch(des1, des2, k=2)
    topResults = []
    for m, n in matches:
        if m.distance < 0.7 * n.distance:
            topResults.append([m])

    return topResults


def compare_images_kaze():
    cwd = os.getcwd()
    target = os.path.join(cwd, 'opencv_target', 'target.png')
    images_list = os.listdir('opencv_images')
    for image in images_list:
        # get my 2 images
        img2 = cv2.imread(target)
        img1 = cv2.imread(os.path.join(cwd, 'opencv_images', image))
        for i in range(0, 360, int(360 / 8)):
            # rotate my image by i
            img_target_rotation = imutils.rotate_bound(img2, i)

            # Initiate KAZE object with default values
            kaze = cv2.KAZE_create()
            kp1, des1 = kaze.detectAndCompute(img1, None)
            kp2, des2 = kaze.detectAndCompute(img2, None)
            matches = calculate_matches(des1, des2)

            try:
                score = 100 * (len(matches) / min(len(kp1), len(kp2)))
            except ZeroDivisionError:
                score = 0
            print(image, score)
            img3 = cv2.drawMatchesKnn(img1, kp1, img_target_rotation, kp2, matches,
                                      None, flags=2)
            img3 = cv2.cvtColor(img3, cv2.COLOR_BGR2RGB)
            plt.imshow(img3)
            plt.show()
            plt.clf()


if __name__ == '__main__':
    compare_images_kaze()

Here's the result of my code:

ex1.png 21.052631578947366
ex2.png 0.0
ex3.png 42.10526315789473

enter image description here enter image description here enter image description here

It does alright! It was able to tell that ex1 is similar and ex2 is not similar, however it states that ex3 is similar (even more similar than ex1). Any extra pre-processing or post-processing (maybe ml, assuming ml is actually useful) or just changes I can do to my method that can be done to keep only ex1 as similar and not ex3?

(Note this score I create is something I found online. Not sure if it's an accurate way to go about it)

ADDED MORE EXAMPLES BELOW

Another set of examples:

Here's what I am searching for

enter image description here

I want the above image to be similar to the middle and bottom images (NOTE: I rotate my target image by 45 degrees and compare it to the images below.)

Feature matching (as stated in answers below) were useful in found similarity with the second image, but not the third image (Even after rotating it properly)

enter image description here

enter image description here

enter image description here



from Using opencv to find the most similar image that contains another image

No comments:

Post a Comment