Wednesday, 19 June 2019

Problem setting video frame rate using AVAssetWriter/AVAssetReader

Situation:

I am trying to export video with some parameters like video bit rate, audio bit rate, frame rate, changing video resolution, etc. Note that I am letting the user set the video frame rate in fractions; like user can set the video frame rate say, 23.98.

I use AVAssetWriter and AVAssetReader for this operation. I use AVAssetWriterInputPixelBufferAdaptor for writing the sample buffers.

Everything else works just fine, except: the video frame rate.

What I have tried:

  1. Setting the AVAssetWriter.movieTimeScale as suggested here. Which does not work. It does not matter whether or not I keep it there; it always takes the source video frame rate. (gist here)

  1. Setting AVVideoExpectedSourceFrameRateKey. Which does not help. (gist here)

  1. Setting AVAssetWriterInput.mediaTimeScale. (gist here)

  1. Using AVAssetReaderVideoCompositionOutput and setting AVMutableVideoComposition.frameDuration; just like SDAVAssetExportSession does. Ironically with SDAVAssetExportSession code, the video is being exported just at the right frame rate what I want, but it just does not work in my code. gist here

I am not sure why it won't work with my code. The issue with this approach is it always returns nil from AVAssetReaderVideoCompositionOutput.copyNextSampleBuffer().


  1. Manually changing the frames timestamp with CMSampleTimingInfo, as suggested here Something like:
var sampleTimingInfo = CMSampleTimingInfo()
var sampleBufferToWrite: CMSampleBuffer?

CMSampleBufferGetSampleTimingInfo(vBuffer, at: 0, timingInfoOut: &sampleTimingInfo)

sampleTimingInfo.duration = CMTimeMake(value: 100, timescale: Int32(videoConfig.videoFrameRate * 100))

sampleTimingInfo.presentationTimeStamp = CMTimeAdd(previousPresentationTimeStamp, sampleTimingInfo.duration)

previousPresentationTimeStamp = sampleTimingInfo.presentationTimeStamp

let status = CMSampleBufferCreateCopyWithNewTiming(allocator: kCFAllocatorDefault, sampleBuffer: vBuffer,sampleTimingEntryCount: 1, sampleTimingArray: &sampleTimingInfo, sampleBufferOut: &sampleBufferToWrite)

With this approach, I do get the frame rate set just right, but it increases the video duration (as mentioned in the comment of the answer). I think at some point I may have to discard some frames (if the target frame rate is lower; I need to lower the frame rate in most of the cases).

If I know that I want 30fps, and my current frame rate is 60fps, it simple to discard every second frame and setting the SampleBuffer time accordingly.

If I go with this approach, how do I decide which frame to discard and if the target frame rate is higher, which frame to duplicate? Reminder: the frame rate could be in fractions.




from Problem setting video frame rate using AVAssetWriter/AVAssetReader

No comments:

Post a Comment