opencv performance degraded on mac osx bigsur

I am using Python 2.7.1 to run opencv via python, on MAC OSX BigSur 11.0.1 ( i9 intel ). The last stable implementation that works on my mac is opencv-python 4.1.2.30 . Since the update to BigSur, I've noticed that the performance of the below opencv function has significantly reduced:

cv2.waitKey(1)

Here is my entire script to playback a video. Note: The video was captured at 130FPS.

import cv2
import sys
import time

video = "myvideo.mp4"

view_name = "my_video"
cv2.namedWindow(view_name)
cv2.moveWindow(view_name,250,150)

cap = cv2.VideoCapture(video)
tots = cap.get(cv2.CAP_PROP_FRAME_COUNT)
i = 0
cv2.createTrackbar('S',view_name, 0,int(tots)-1, lambda x:x)
cv2.setTrackbarPos('S',view_name,0)

status = 'pause'
previous_status = 'pause'

while True:
  try:
    if i==tots-1:
      i=0
    cap.set(cv2.CAP_PROP_POS_FRAMES, i)
    #start = time.time()
    ret, im = cap.read()
    cv2.imshow(view_name, im)
    status = { ord(' '):'play_pause_click',
            ord('a'):'prev_frame', ord('A'):'prev_frame',
            ord('d'):'next_frame', ord('D'):'next_frame',
            -1: status, 
            27: 'exit'}[cv2.waitKey(1)]
     #checkpointTwo = time.time()
     #print(checkpointTwo - start)

    if (status == 'play_pause_click' and previous_status == 'play_pause_click'):
        status = 'pause'
        previous_status = 'pause'
    if (status == 'play_pause_click'):
        status = 'play'
        previous_status = 'play_pause_click'

    if status == 'play':
      i+=1
      cv2.setTrackbarPos('S',view_name,i)
      continue
    if status == 'pause':
      i = cv2.getTrackbarPos('S',view_name)
    if status == 'exit':
        break
    if status=='prev_frame':
        i-=1
        cv2.setTrackbarPos('S',view_name,i)
        status='pause'
    if status=='next_frame':
        i+=1
        cv2.setTrackbarPos('S',view_name,i)
        status='pause'

  except KeyError:
    print("Invalid Key was pressed")
cv2.destroyWindow(view_name)

I click the spacebar on my machine to start playing the video frame by frame automatically. I take note of the elapsed time, and I can see these results:

At the 90th percentile, nearly 150ms is taken for the code block to execute. For comparison, if I measure cap.read() and cv2.imshow() functions, my execution time is < 1 ms. Here is a cdf plot:

While I have no benchmarks while using previous MAC OSX versions, I can visually tell it is significantly slower.

I've looked on the opencv-python documentation page, which has this quote:

Unofficial pre-built CPU-only OpenCV packages for Python.

I am starting to suspect that this may be the issue and that I should re-build the opencv package from scratch for my machine. However, I am not sure if that will solve the problem or how to perform this from scratch. It is also entirely possible that I might be using the opencv function incorrectly.

What can I do to fix the large 150ms?

from opencv performance degraded on mac osx bigsur