I'm not really sure how to go about asking this question, I would appreciate any helpful feedback on improving this question, I would've preferred to not post a question, but I've had this issue for several weeks now and I have not been able to solve it. I will probably post a bounty when I am eligible to do so. I am trying to make a function that accepts a video URL as input (local video), in turn this function tries to create a video with a blurry background, with the original video at its center and scaled down. My issue is that my code is working fine, aside from when I use videos that are directly recorded from the iPhone camera.
An example of what I am trying to achieve is the following (Taken from my code):
The input video here is an mp4. I have been able to make the code work as well with mov files that I've downloaded online. But when I use mov files recorded from the iOS camera, I end up with the following:
(How can i post pictures that take less space in the question?)
Now, the reason I am not sure how to ask this question is because there is a fair amount of code in the process and I haven't been able to fully narrow down the question but I believe it is in the function that I will paste below. I will also post a link to a github repository, where a barebones version of my project has been posted for anyone curious or willing to help. I must confess that the code I am using was originally written by a StackOverflow user named TheTiger on the following question: AVFoundation - Adding blur background to video . I've refactored segments of this, and with their permission, was allowed to post the question here.
My github repo is linked here: GITHUB REPO My demo is set up with 3 different videos, an mp4 downloaded from the web (working), an mov downloaded from the web (Working) and an mov I've recorded on my phone (not working)
The code I imagine is causing the issue is here:
fileprivate func addAllVideosAtCenterOfBlur(asset: AVURLAsset, blurVideo: AVURLAsset, scale: CGFloat, completion: @escaping BlurredBackgroundManagerCompletion) {
let mixComposition = AVMutableComposition()
var instructionLayers : Array<AVMutableVideoCompositionLayerInstruction> = []
let blurVideoTrack = mixComposition.addMutableTrack(withMediaType: AVMediaType.video, preferredTrackID: kCMPersistentTrackID_Invalid)
if let videoTrack = blurVideo.tracks(withMediaType: AVMediaType.video).first {
let timeRange = CMTimeRange(start: .zero, duration: blurVideo.duration)
try? blurVideoTrack?.insertTimeRange(timeRange, of: videoTrack, at: .zero)
}
let timeRange = CMTimeRange(start: .zero, duration: asset.duration)
let track = mixComposition.addMutableTrack(withMediaType: AVMediaType.video, preferredTrackID: kCMPersistentTrackID_Invalid)
if let videoTrack = asset.tracks(withMediaType: AVMediaType.video).first {
try? track?.insertTimeRange(timeRange, of: videoTrack, at: .zero)
let layerInstruction = AVMutableVideoCompositionLayerInstruction(assetTrack: track!)
let properties = scaleAndPositionInAspectFitMode(forTrack: videoTrack, inArea: size, scale: scale)
let videoOrientation = videoTrack.getVideoOrientation()
let assetSize = videoTrack.assetSize()
let preferredTransform = getPreferredTransform(videoOrientation: videoOrientation, assetSize: assetSize, defaultTransform: asset.preferredTransform, properties: properties)
layerInstruction.setTransform(preferredTransform, at: .zero)
instructionLayers.append(layerInstruction)
}
/// Adding audio
if let audioTrack = asset.tracks(withMediaType: AVMediaType.audio).first {
let aTrack = mixComposition.addMutableTrack(withMediaType: AVMediaType.audio, preferredTrackID: kCMPersistentTrackID_Invalid)
try? aTrack?.insertTimeRange(timeRange, of: audioTrack, at: .zero)
}
/// Blur layer instruction
let layerInstruction = AVMutableVideoCompositionLayerInstruction(assetTrack: blurVideoTrack!)
instructionLayers.append(layerInstruction)
let mainInstruction = AVMutableVideoCompositionInstruction()
mainInstruction.timeRange = timeRange
mainInstruction.layerInstructions = instructionLayers
let mainCompositionInst = AVMutableVideoComposition()
mainCompositionInst.instructions = [mainInstruction]
mainCompositionInst.frameDuration = CMTimeMake(value: 1, timescale: 30)
mainCompositionInst.renderSize = size
//let url = URL(fileURLWithPath: "/Users/enacteservices/Desktop/final_video.mov")
let url = self.videoOutputUrl(filename: "finalBlurred")
try? FileManager.default.removeItem(at: url)
performExport(composition: mixComposition, instructions: mainCompositionInst, stage: 2, outputUrl: url) { (error) in
if let error = error {
completion(nil, error)
} else {
completion(url, nil)
}
}
}
The getPreferredTransform() function is also quite relevant:
fileprivate func getPreferredTransform(videoOrientation: UIImage.Orientation, assetSize: CGSize, defaultTransform: CGAffineTransform, properties: Properties) -> CGAffineTransform {
switch videoOrientation {
case .down:
return handleDownOrientation(assetSize: assetSize, defaultTransform: defaultTransform, properties: properties)
case .left:
return handleLeftOrientation(assetSize: assetSize, defaultTransform: defaultTransform, properties: properties)
case .right:
return handleRightOrientation(properties: properties)
case .up:
return handleUpOrientation(assetSize: assetSize, defaultTransform: defaultTransform, properties: properties)
default:
return handleOtherCases(assetSize: assetSize, defaultTransform: defaultTransform, properties: properties)
}
}
fileprivate func handleDownOrientation(assetSize: CGSize, defaultTransform: CGAffineTransform, properties: Properties) -> CGAffineTransform {
let rotateTransform = CGAffineTransform(rotationAngle: -CGFloat(Double.pi/2.0))
// Scale
let scaleTransform = CGAffineTransform(scaleX: properties.scale.width, y: properties.scale.height)
// Translate
var ytranslation: CGFloat = assetSize.height
var xtranslation: CGFloat = 0
if properties.position.y == 0 {
xtranslation = -(assetSize.width - ((size.width/size.height) * assetSize.height))/2.0
}
else {
ytranslation = assetSize.height - (assetSize.height - ((size.height/size.width) * assetSize.width))/2.0
}
let translationTransform = CGAffineTransform(translationX: xtranslation, y: ytranslation)
// Final transformation - Concatination
let finalTransform = defaultTransform.concatenating(rotateTransform).concatenating(translationTransform).concatenating(scaleTransform)
return finalTransform
}
fileprivate func handleLeftOrientation(assetSize: CGSize, defaultTransform: CGAffineTransform, properties: Properties) -> CGAffineTransform {
let rotateTransform = CGAffineTransform(rotationAngle: -CGFloat(Double.pi))
// Scale
let scaleTransform = CGAffineTransform(scaleX: properties.scale.width, y: properties.scale.height)
// Translate
var ytranslation: CGFloat = assetSize.height
var xtranslation: CGFloat = assetSize.width
if properties.position.y == 0 {
xtranslation = assetSize.width - (assetSize.width - ((size.width/size.height) * assetSize.height))/2.0
} else {
ytranslation = assetSize.height - (assetSize.height - ((size.height/size.width) * assetSize.width))/2.0
}
let translationTransform = CGAffineTransform(translationX: xtranslation, y: ytranslation)
// Final transformation - Concatination
let finalTransform = defaultTransform.concatenating(rotateTransform).concatenating(translationTransform).concatenating(scaleTransform)
return finalTransform
}
fileprivate func handleRightOrientation(properties: Properties) -> CGAffineTransform {
let scaleTransform = CGAffineTransform(scaleX: properties.scale.width, y: properties.scale.height)
// Translate
let translationTransform = CGAffineTransform(translationX: properties.position.x, y: properties.position.y)
let finalTransform = scaleTransform.concatenating(translationTransform)
return finalTransform
}
fileprivate func handleUpOrientation(assetSize: CGSize, defaultTransform: CGAffineTransform, properties: Properties) -> CGAffineTransform {
return handleOtherCases(assetSize: assetSize, defaultTransform: defaultTransform, properties: properties)
}
fileprivate func handleOtherCases(assetSize: CGSize, defaultTransform: CGAffineTransform, properties: Properties) -> CGAffineTransform {
let rotateTransform = CGAffineTransform(rotationAngle: CGFloat(Double.pi/2.0))
let scaleTransform = CGAffineTransform(scaleX: properties.scale.width, y: properties.scale.height)
var ytranslation: CGFloat = 0
var xtranslation: CGFloat = assetSize.width
if properties.position.y == 0 {
xtranslation = assetSize.width - (assetSize.width - ((size.width/size.height) * assetSize.height))/2.0
}
else {
ytranslation = -(assetSize.height - ((size.height/size.width) * assetSize.width))/2.0
}
let translationTransform = CGAffineTransform(translationX: xtranslation, y: ytranslation)
let finalTransform = defaultTransform.concatenating(rotateTransform).concatenating(translationTransform).concatenating(scaleTransform)
return finalTransform
}
from Creating AVComposition with a Blurry Background in iOS
No comments:
Post a Comment