Sunday, 9 May 2021

Preprocessing a video in android for pytorch

What is the best way to preprocess video data in Android Kotlin, in preparation to feed into a PyTorch Android model? Specifically, I have a ready made model in PyTorch, and I've converted it to be ready for PyTorch Mobile.

During training, the model takes in raw footage from phones, and is preprocessed to (1) be greyscale, and (2) compressed to a specific smaller resolution that I have specified, (3) converted to a Tensor to be fed into a neural net (or potentially send the compressed video to remote server). I use OpenCV for this, but I'm wondering what the easiest way to do this would be in Android Kotlin?

Python code for reference:


def save_video(filename):

    frames = []

    cap = cv2.VideoCapture(filename)
    frameCount = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
    frameWidth = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
    frameHeight = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))

    buf_c = np.empty((frameCount, frameHeight, frameWidth, 3), np.dtype('uint8'))
    buf = np.empty((frameCount, frameHeight, frameWidth), np.dtype('uint8'))

    fc = 0
    ret = True

    # 9:16 ratio
    width = 121
    height = 216
    dim = (width, height)

    # Loop until the end of the video
    while fc < frameCount and ret:
        ret, buf_c[fc] = cap.read()

        # convert to greyscale
        buf[fc] = cv2.cvtColor(buf_c[fc], cv2.COLOR_BGR2GRAY)

        # reduce resolution
        resized = cv2.resize(buf[fc], dim, interpolation = cv2.INTER_AREA)

        frames.append(resized)
        fc += 1

    # release the video capture object
    cap.release()

    # Closes all the windows currently opened.
    cv2.destroyAllWindows()

    return frames


from Preprocessing a video in android for pytorch

No comments:

Post a Comment