Sunday 27 September 2020

FFMPEG H264 with custom overlay per frame

We have a stream that is stored in the cloud (Amazon S3) as individual H264 frames. The frames are stored as framexxxxxx.264, the numbering doesn't start from 0 but rather from some larger number, say 1000 (so, frame001000.264)

The goal is to create a mp4 clip which is either timelapse or just faster for inspection and other checking (much faster, compressing around 3 hours of video down to < 20 minutes), this also requires we overlay the frame number (the filename) on the frame itself

At first I was creating a timelapse by pulling from S3 only the keyframes (i-frames? still rather new to codecs & stuff) and overlaying the filename on them and saving as png (which probably isn't needed, but that's what I did) using (this command is used inside a python script)

ffmpeg -y -i {h264_name} -vf \"scale=1920:-1, 
drawtext=fontfile=/usr/share/fonts/truetype/ubuntu-font-family/Ubuntu-B.ttf:fontsize=34:text={txt}:fontcolor=white:x=50:y=50:bordercolor=black:borderw=2\" 
-c:a copy -pix_fmt yuv420p {basename}.png

after this I combined all the frames by using python to convert the lowest numbered frame to 0.png and incrementing (so it would be continuous, because I only used keyframes the numbers originally weren't sequential) and running

ffmpeg -y -f image2 -i %d.png -r {self.params.fps} -vcodec libx264 -crf {self.params.crf} -pix_fmt yuv420p {out_file}

and this worked great, but the difference between keyframes was too long to allow for proper inspection

so now for the question(s)

since I know frames that are not keyframes (p-frames?) can't be used alone by ffmpeg, the method of overlaying the file name and converting it to png (or keep as h264, same thing) won't work, or at least, I couldn't find a way for it to work, maybe there's a way to specify a frame's keyframe?, how can one overlay the filename (and not the frame number as shown here for example)

Also, is it possible to skip some p-frames between the keyframes? (so if a keyframe is every 30 frames, we would take a keyframe, a frame 15 frames later, and next another keyframe)

I thought about using ffmpeg's pipe option to feed it with the files as they're being downloaded, but I'm not sure if I can specify drawtext this way

Also, if there's another alternative that can achieve that (at first I was converting to png, using python and OpenCV to add the filename and then merging the pngs to mp4, but then I found drawtext can do that in a single command so I used it)



from FFMPEG H264 with custom overlay per frame

No comments:

Post a Comment