Android - Rotate video frames before sending to Wowza Streaming Engine using WebRTC

I want to stream video from android camera to Wowza Streaming Engine (WSE) using WebRTC. When device in landscape mode, everything work well. Then I try to stream by putting the device in portrait mode.

The first thing I notice in WSE player is the video stream has been rotated 90 counter-clockwise. I figured out that WebRTC do not rotate each video frame that comes out from onPreviewFrame API before sending to WSE and unfortunately WSE do not support any mechanisms to rotate video frame at their side at least so far.

So I checked out WebRTC android native source code and modify it to rotate each video frame before sending to WSE. And right now I can see the video stream in portrait mode in WSE player.

But it has an issue is sometimes video stream look so weird. Please see the following images.

A normal image

A weird image

I put camera to a fix position. At the first time WSE player displays the first one, but sometimes the second one is showed up.

And here is the file in WebRTC source code that I changed. ~/webrtc/src/sdk/android/src/jni/androidvideotracksource.cc

void AndroidVideoTrackSource::OnByteBufferFrameCaptured(const void* frame_data,
                                                        int length,
                                                        int width,
                                                        int height,
                                                        VideoRotation rotation,
                                                        int64_t timestamp_ns) {
  RTC_DCHECK(camera_thread_checker_.CalledOnValidThread());

  int64_t camera_time_us = timestamp_ns / rtc::kNumNanosecsPerMicrosec;
  int64_t translated_camera_time_us =
      timestamp_aligner_.TranslateTimestamp(camera_time_us, rtc::TimeMicros());

  int adapted_width;
  int adapted_height;
  int crop_width;
  int crop_height;
  int crop_x;
  int crop_y;

  if (!AdaptFrame(width, height, camera_time_us, &adapted_width,
                  &adapted_height, &crop_width, &crop_height, &crop_x,
                  &crop_y)) {
    return;
  }

  const uint8_t* y_plane = static_cast<const uint8_t*>(frame_data);
  const uint8_t* uv_plane = y_plane + width * height;
  const int uv_width = (width + 1) / 2;

  RTC_CHECK_GE(length, width * height + 2 * uv_width * ((height + 1) / 2));

  // Can only crop at even pixels.
  crop_x &= ~1;
  crop_y &= ~1;
  // Crop just by modifying pointers.
  y_plane += width * crop_y + crop_x;
  uv_plane += uv_width * crop_y + crop_x;

  rtc::scoped_refptr<I420Buffer> buffer =
      buffer_pool_.CreateBuffer(adapted_width, adapted_height);

  nv12toi420_scaler_.NV12ToI420Scale(
      y_plane, width, uv_plane, uv_width * 2, crop_width, crop_height,
      buffer->MutableDataY(), buffer->StrideY(),
      // Swap U and V, since we have NV21, not NV12.
      buffer->MutableDataV(), buffer->StrideV(), buffer->MutableDataU(),
      buffer->StrideU(), buffer->width(), buffer->height());

  // TODO: Rotate I420 frame 90 degrees clockwise.
  rtc::scoped_refptr<I420Buffer> rotated_buffer =
      I420Buffer::Rotate(*buffer, kVideoRotation_90);

  OnFrame(VideoFrame(rotated_buffer, rotation, translated_camera_time_us));
}

I added this line of code to rotate I420 frame 90 degrees clockwise.

// TODO: Rotate I420 frame 90 degrees clockwise.
  rtc::scoped_refptr<I420Buffer> rotated_buffer =
      I420Buffer::Rotate(*buffer, kVideoRotation_90);

I would appreciate any help!

from Android - Rotate video frames before sending to Wowza Streaming Engine using WebRTC