What is the best way to preprocess video data in Android Kotlin, in preparation to feed into a PyTorch Android model? Specifically, I have a ready made model in PyTorch, and I've converted it to be ready for PyTorch Mobile.
During training, the model takes in raw footage from phones, and is preprocessed to (1) be greyscale, and (2) compressed to a specific smaller resolution that I have specified, (3) converted to a Tensor to be fed into a neural net (or potentially send the compressed video to remote server). I use OpenCV for this, but I'm wondering what the easiest way to do this would be in Android Kotlin?
Python code for reference:
def save_video(filename):
frames = []
cap = cv2.VideoCapture(filename)
frameCount = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
frameWidth = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
frameHeight = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
buf_c = np.empty((frameCount, frameHeight, frameWidth, 3), np.dtype('uint8'))
buf = np.empty((frameCount, frameHeight, frameWidth), np.dtype('uint8'))
fc = 0
ret = True
# 9:16 ratio
width = 121
height = 216
dim = (width, height)
# Loop until the end of the video
while fc < frameCount and ret:
ret, buf_c[fc] = cap.read()
# convert to greyscale
buf[fc] = cv2.cvtColor(buf_c[fc], cv2.COLOR_BGR2GRAY)
# reduce resolution
resized = cv2.resize(buf[fc], dim, interpolation = cv2.INTER_AREA)
frames.append(resized)
fc += 1
# release the video capture object
cap.release()
# Closes all the windows currently opened.
cv2.destroyAllWindows()
return frames
from Preprocessing a video in android for pytorch
No comments:
Post a Comment