Monday, 30 July 2018

FFMpeg frame drop when merging a image sequence and overlaying during video creation process

I have reviewed both the answer as mentioned in Similar Issue But still, I am having some frame drops.
Here is what I was doing : Question 1 Question 2
Currently, I am showing a few images (one after another each with duration 'x') on a sample background along with a music playing in the background.
This is the command I am using to get the desired functionality:
ffmpeg 
-hide_banner 
-y 
-loop 1 
-r 1 
-i /data/user/0/mojo.wed.com.wedmojo/cache/wedding_video/Images/IMG_WED_MOJO_WEDDING_FRAME
.png 
-r 1/4 
-i /data/user/0/mojo.wed.com.wedmojo/cache/wedding_video/Images/IMG_WED_MOJO_SELECTED_IMAGE
_%d.png 
-filter_complex overlay=(W-w)/2:(H-h)/2,drawtext=text=WedMojo:fontcolor=white:fontfile=/
data/user/0/mojo.wed.com.wedmojo/cache/wedding_video/Files/FONT_WED_MOJO_WEDDING.ttf:
fontsize=30:x=w-tw-20:y=h-th-20 
-c:v libx264 
-profile:v high 
-crf 17 -pix_fmt yuv420p 
-preset ultrafast 
-t 16 -strict experimental 
-r 24 
/data/user/0/mojo.wed.com.wedmojo/cache/wedding_video/Videos/VID_WED_MOJO_WEDDING_OUTPUT.mp4

note: -r 1/4 for each image to be shown for 4 sec.
I have checked the format of the images. All are the same. Also on changing preset and other values, the problem seems to be same. Log states drop=0, however, there are gaps as the overlay image is removed in between.
Video sample with the duration set to 4 i.e each image to be set for 4 seconds.
-> -> Video Output with 4 sec duration for each image
However, there are no frame drops when the duration set to 1 or 2.
-> -> Video Output with 2 sec duration for each image, ignore zoompan
Log details:
D/CreateVideoActivity: COMMAND: -hide_banner -y -loop 1 -r 1 -i /data/user/0/mojo.wed.com.
wedmojo/cache/wedding_video/Images/IMG_WED_MOJO_WEDDING_FRAME.png -r 1/4 -i /data/user/0/
mojo.wed.com.wedmojo/cache/wedding_video/Images/IMG_WED_MOJO_SELECTED_IMAGE_%d.png 
-filter_complex overlay=(W-w)/2:(H-h)/2,drawtext=text=WedMojo:fontcolor=white:fontfile=/
data/user/0/mojo.wed.com.wedmojo/cache/wedding_video/Files/FONT_WED_MOJO_WEDDING.ttf:
fontsize=30:x=w-tw-20:y=h-th-20 -c:v libx264 -profile:v high -crf 17 -pix_fmt yuv420p 
-preset ultrafast -t 16 -strict experimental -r 24 /data/user/0/mojo.wed.com.wedmojo/cache
/wedding_video/Videos/VID_WED_MOJO_WEDDING_OUTPUT.mp4
D/SurfaceUtils: disconnecting from surface 0x8a091008, reason disconnectFromSurface

D/FFmpeg: Running publishing updates method
D/CreateVideoActivity: onProgress() => Input #0, png_pipe, from '/data/user/0/mojo.wed.com.
wedmojo/cache/wedding_video/Images/IMG_WED_MOJO_WEDDING_FRAME.png':
    onProgress() =>   Duration: N/A, bitrate: N/A
    onProgress() =>     Stream #0:0: Video: png, rgb24(pc), 1280x720, 25 fps, 25 tbr, 25 
tbn, 25 tbc
    onProgress() => Input #1, image2, from '/data/user/0/mojo.wed.com.wedmojo/cache/
wedding_video/Images/IMG_WED_MOJO_SELECTED_IMAGE_%d.png':
    onProgress() =>   Duration: 00:00:00.16, start: 0.000000, bitrate: N/A
D/CreateVideoActivity: onProgress() =>     Stream #1:0: Video: png, rgb24(pc), 600x600 
[SAR 1:1 DAR 1:1], 25 fps, 25 tbr, 25 tbn, 25 tbc
    onProgress() => Stream mapping:
    onProgress() =>   Stream #0:0 (png) -> overlay:main
    onProgress() =>   Stream #1:0 (png) -> overlay:overlay
    onProgress() =>   drawtext -> Stream #0:0 (libx264)
    onProgress() => Press [q] to stop, [?] for help
D/CreateVideoActivity: onProgress() => [libx264 @ 0xb0857c00] using cpu capabilities: none!
D/CreateVideoActivity: onProgress() => [libx264 @ 0xb0857c00] profile Constrained Baseline,
 level 3.1
    onProgress() => [libx264 @ 0xb0857c00] 264 - core 152 r2851M ba24899 - H.264/MPEG-4 AVC 
codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 
deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 
chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 
threads=3 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 
constrained_intra=0 bframes=0 weightp=0 keyint=250 keyint_min=24 scenecut=0 intra_refresh=0 
rc=crf mbtree=0 crf=17.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
    onProgress() => Output #0, mp4, to '/data/user/0/mojo.wed.com.wedmojo/cache/wedding_
video/Videos/VID_WED_MOJO_WEDDING_OUTPUT.mp4':
D/CreateVideoActivity: onProgress() =>   Metadata:
D/CreateVideoActivity: onProgress() =>     encoder         : Lavf58.12.100
    onProgress() =>     Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 
1280x720, q=-1--1, 24 fps, 12288 tbn, 24 tbc (default)
    onProgress() =>     Metadata:
D/CreateVideoActivity: onProgress() =>       encoder         : Lavc58.18.100 libx264
    onProgress() =>     Side data:
    onProgress() =>       cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
D/CreateVideoActivity: onProgress() => frame=   96 fps=0.0 q=6.0 size=     
256kB time=00:00:03.79 bitrate= 553.2kbits/s dup=93 drop=0 speed=3.94x    
D/CreateVideoActivity: onProgress() => frame=  120 fps= 79 q=6.0 size=    
 512kB time=00:00:04.79 bitrate= 875.4kbits/s dup=116 drop=0 speed=3.13x    
D/CreateVideoActivity: onProgress() => frame=  192 fps= 91 q=6.0 size=     
512kB time=00:00:07.79 bitrate= 538.4kbits/s dup=185 drop=0 speed=3.68x    
D/CreateVideoActivity: onProgress() => frame=  216 fps= 79 q=6.0 size=    
 768kB time=00:00:08.79 bitrate= 715.7kbits/s dup=208 drop=0 speed=3.21x    
D/CreateVideoActivity: onProgress() => frame=  288 fps= 88 q=6.0 size=   
 1024kB time=00:00:11.79 bitrate= 711.4kbits/s dup=278 drop=0 speed= 3.6x    
D/CreateVideoActivity: onProgress() => frame=  312 fps= 82 q=6.0 size=    
1536kB time=00:00:12.79 bitrate= 983.7kbits/s dup=301 drop=0 speed=3.37x    
D/CreateVideoActivity: onProgress() => frame=  384 fps= 86 q=6.0 size=   
 1536kB time=00:00:15.79 bitrate= 796.8kbits/s dup=370 drop=0 speed=3.52x    
D/CreateVideoActivity: onProgress() => frame=  384 fps= 78 q=-1.0 Lsize=    
1783kB time=00:00:15.95 bitrate= 915.4kbits/s dup=393 drop=0 speed=3.24x    
D/CreateVideoActivity: onProgress() => video:1781kB audio:0kB subtitle:0kB other 
streams:0kB global headers:0kB muxing overhead: 0.130068%
D/CreateVideoActivity: onProgress() => [libx264 @ 0xb0857c00] frame I:2    
 Avg QP: 9.00  size:152054
    onProgress() => [libx264 @ 0xb0857c00] frame P:382   Avg QP: 6.69  size:  3976
    onProgress() => [libx264 @ 0xb0857c00] mb I  I16..4: 100.0%  0.0%  0.0%
    onProgress() => [libx264 @ 0xb0857c00] mb P  I16..4:  0.5%  0.0%  0.0%  P16..4: 10.7% 
 0.0%  0.0%  0.0%  0.0%    skip:88.7%
    onProgress() => [libx264 @ 0xb0857c00] coded y,uvDC,uvAC intra: 56.4% 68.7% 52.5% 
inter: 4.2% 2.0% 1.5%
    onProgress() => [libx264 @ 0xb0857c00] i16 v,h,dc,p: 49% 25% 15% 11%
    onProgress() => [libx264 @ 0xb0857c00] i8c dc,h,v,p: 38% 28% 23% 11%
    onProgress() => [libx264 @ 0xb0857c00] kb/s:911.53
D/CreateVideoActivity: onSuccess() => Input #0, png_pipe, from '/data/user/0/mojo.wed.com.
wedmojo/cache/wedding_video/Images/IMG_WED_MOJO_WEDDING_FRAME.png':
      Duration: N/A, bitrate: N/A
        Stream #0:0: Video: png, rgb24(pc), 1280x720, 25 fps, 25 tbr, 25 tbn, 25 tbc
    Input #1, image2, from '/data/user/0/mojo.wed.com.wedmojo/cache/wedding_video/Images/
IMG_WED_MOJO_SELECTED_IMAGE_%d.png':
      Duration: 00:00:00.16, start: 0.000000, bitrate: N/A
        Stream #1:0: Video: png, rgb24(pc), 600x600 [SAR 1:1 DAR 1:1], 25 fps, 25 tbr, 25 
tbn, 25 tbc
    Stream mapping:
      Stream #0:0 (png) -> overlay:main
      Stream #1:0 (png) -> overlay:overlay
      drawtext -> Stream #0:0 (libx264)
    Press [q] to stop, [?] for help
    [libx264 @ 0xb0857c00] using cpu capabilities: none!
    [libx264 @ 0xb0857c00] profile Constrained Baseline, level 3.1
    [libx264 @ 0xb0857c00] 264 - core 152 r2851M ba24899 - H.264/MPEG-4 AVC codec - Copyleft
 2003-2017 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0
 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 
trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=3 lookahead_
threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0
 bframes=0 weightp=0 keyint=250 keyint_min=24 scenecut=0 intra_refresh=0 rc=crf mbtree=0 
crf=17.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
    Output #0, mp4, to '/data/user/0/mojo.wed.com.wedmojo/cache/wedding_video/Videos/
VID_WED_MOJO_WEDDING_OUTPUT.mp4':
      Metadata:
        encoder         : Lavf58.12.100
        Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 1280x720, q=-1--1,
 24 fps, 12288 tbn, 24 tbc (default)
        Metadata:
          encoder         : Lavc58.18.100 libx264
        Side data:
          cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
    frame=   96 fps=0.0 q=6.0 size=     256kB time=00:00:03.79 bitrate= 553.2kbits/s dup=93
 drop=0 speed=3.94x    
    frame=  120 fps= 79 q=6.0 size=     512kB time=00:00:04.79 bitrate= 875.4kbits/s dup=116
 drop=0 speed=3.13x    
    frame=  192 fps= 91 q=6.0 size=     512kB time=00:00:07.79 bitrate= 538.4kbits/s dup=185
 drop=0 speed=3.68x    
    frame=  216 fps= 79 q=6.0 size=     768kB time=00:00:08.79 bitrate= 715.7kbits/s dup=208
 drop=0 speed=3.21x    
    frame=  288 fps= 88 q=6.0 size=    1024kB time=00:00:11.79 bitrate= 711.4kbits/s dup=278
 drop=0 speed= 3.6x    
    frame=  312 fps= 82 q=6.0 size=    1536kB time=00:00:12.79 bitrate= 983.7kbits/s dup=301
 drop=0 speed=3.37x    
    frame=  384 fps= 86 q=6.0 size=    1536kB time=00:00:15.79 bitrate= 796.8kbits/s dup=370
 drop=0 speed=3.52x    
    frame=  384 fps= 78 q=-1.0 Lsize=    1783kB time=00:00:15.95 bitrate= 915.4kbits/s 
dup=393 drop=0 speed=3.24x    
    video:1781kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing
 overhead: 0.130068%
    [libx264 @ 0xb0857c00] frame I:2     Avg QP: 9.00  size:152054
    [libx264 @ 0xb0857c00] frame P:382   Avg QP: 6.69  size:  3976
    [libx264 @ 0xb0857c00] mb I  I16..4: 100.0%  0.0%  0.0%
    [libx264 @ 0xb0857c00] mb P  I16..4:  0.5%  0.0%  0.0%  P16..4: 10.7%  0.0%  0.0%  0.0% 
 0.0%    skip:88.7%
    [libx264 @ 0xb0857c00] coded y,uvDC,uvAC intra: 56.4% 68.7% 52.5% inter: 4.2% 2.0% 1.5%
    [libx264 @ 0xb0857c00] i16 v,h,dc,p: 49% 25% 15% 11%
    [libx264 @ 0xb0857c00] i8c dc,h,v,p: 38% 28% 23% 11%
    [libx264 @ 0xb0857c00] kb/s:911.53
I/OMXClient: Treble IOmx obtained
D/SurfaceUtils: connecting to surface 0x8c6ef008, reason connectToSurface
I/MediaCodec: [OMX.google.h264.decoder] setting surface generation to 23113736
D/SurfaceUtils: disconnecting from surface 0x8c6ef008, reason connectToSurface(reconnect)
D/SurfaceUtils: connecting to surface 0x8c6ef008, reason connectToSurface(reconnect)
E/ACodec: [OMX.google.h264.decoder] setPortMode on output to DynamicANWBuffer failed w/ 
err -1010
I/ACodec: codec does not support config priority (err -1010)
D/MediaCodec: [OMX.google.h264.decoder] setting dataspace on output surface to #104
D/SoftwareRenderer: setting dataspace on output surface to #104

Let me know if any details required from my side. Your valuable suggestions are welcomed.


from FFMpeg frame drop when merging a image sequence and overlaying during video creation process

No comments:

Post a Comment