How can I identify rectangles in an image when they are of different colours, outlines and sometimes very close to the background colour

I'm trying to extract rectangles from an image. These are digital stickies on a digital notepad. They can be any user configurable colour, including transparent with a border. I want to be able to input a jpg/png file and get back a list of each of the rectangles, their coordinates and the colour of the rectangle.

OpenCV with Python is the route that I want to use for this. Below is the example image, the intention is to detect all of the rectangles only and retrieve the above mentioned information.

I've done quite a lot of reading and been using the find contours method to try and achieve my goal however I'm not getting the desired result.

import cv2

# reading image
img = cv2.imread('images/example_shapes.jpg')

# converting image into grayscale image
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

# setting threshold of gray image
_, threshold = cv2.threshold(gray, 127, 255, cv2.THRESH_BINARY)

# using a findContours() function
contours, _ = cv2.findContours(
    threshold, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)

i = 0

# list for storing names of shapes
for contour in contours:

    # here we are ignoring first counter because
    # findcontour function detects whole image as shape
    if i == 0:
        i = 1
        continue

    # cv2.approxPloyDP() function to approximate the shape
    approx = cv2.approxPolyDP(
        contour, 0.01 * cv2.arcLength(contour, True), True)

    if len(approx) == 4:
        cv2.drawContours(img, [contour], 0, (0, 0, 255), 5)

# displaying the image after drawing contours
# img = cv2.resize(img, (500, 500))
cv2.imshow('shapes', img)

cv2.waitKey(0)
cv2.destroyAllWindows()

This would only detect the 2 rectangles in the middle and gave the following:

I had then attempted to adjust the threshold to be adaptive thresholding:

threshold = cv2.adaptiveThreshold(gray, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY, 13, 7)

which produced the following result

Neither approach seems to be able to detect the rectangles who are close together and also a close colour to the backround and neither detects the rectangles with a stroke. The adaptive thresholding also returns a lot of items that are irrelevant.

Any suggestions on how to approach would be very welcome!

from How can I identify rectangles in an image when they are of different colours, outlines and sometimes very close to the background colour