Saturday 22 August 2020

Gstreamer plugin cannot play video correctly on Android when using udpscr

I'm having some issue while implementing a gstreamer plugin to play RTP video on Android. I have the following code (which works correctly):

    full_pipeline_description = g_strdup_printf("playbin3 uri=%s", uri);
gub_log_pipeline(pipeline, "Using pipeline: %s", full_pipeline_description);
pipeline->pipeline = gst_parse_launch(full_pipeline_description, &err);
g_free(full_pipeline_description);
if (err) {
    gub_log_pipeline(pipeline, "Failed to create pipeline: %s", err->message);
    return;
}

vsink = gst_parse_bin_from_description(gub_get_video_branch_description(), TRUE, NULL);
gub_log_pipeline(pipeline, "Using video sink: %s", gub_get_video_branch_description());
g_object_set(pipeline->pipeline, "video-sink", vsink, NULL);
g_object_set(pipeline->pipeline, "flags", 0x0003, NULL);

bus = gst_pipeline_get_bus(GST_PIPELINE(pipeline->pipeline));
gst_bus_add_signal_watch(bus);
gst_object_unref(bus);
g_signal_connect(bus, "message", G_CALLBACK(message_received), pipeline);

if (vsink) {
    // Plant a pad probe to answer context queries
    GstElement *sink;
    sink = gst_bin_get_by_name(GST_BIN(vsink), "sink");
    if (sink) {
        GstPad *pad = gst_element_get_static_pad(sink, "sink");
        if (pad) {
            gulong id = gst_pad_add_probe(pad, GST_PAD_PROBE_TYPE_QUERY_DOWNSTREAM, pad_probe, pipeline, NULL);
            gst_object_unref(pad);
        }
        gst_object_unref(sink);
    }
}

And using the same code with another pipeline (based on udpsrc instead of playbin3) does not. The pipeline I use in this case is:

udpsrc port=53512 ! application/x-rtp,media=video,clock-rate=90000,encoding-name=H264,payload=96 ! rtph264depay ! decodebin3 ! glupload ! glcolorconvert ! video/x-raw(memory:GLMemory),format=RGBA,texture-target=2D ! fakesink sync=0 qos=1 name=sink

The code is as following:

      full_pipeline_description = g_strdup_printf("%s", pipeline_cmd);
  gub_log_pipeline(pipeline, "Using pipeline: %s", full_pipeline_description);
  pipeline->pipeline = gst_parse_launch(full_pipeline_description, &err);
  g_free(full_pipeline_description);
  if (err) {
    gub_log_pipeline(pipeline, "Failed to create pipeline: %s", err->message);
    return;
  }

  vsink = gst_parse_bin_from_description(gub_get_video_branch_description(), TRUE, NULL);
  gub_log_pipeline(pipeline, "Using video sink: %s", gub_get_video_branch_description());
  g_object_set(pipeline->pipeline, "sink", vsink, NULL);
  g_object_set(pipeline->pipeline, "flags", 0x0003, NULL);

  bus = gst_pipeline_get_bus(GST_PIPELINE(pipeline->pipeline));
  gst_bus_add_signal_watch(bus);
  gst_object_unref(bus);
  g_signal_connect(bus, "message", G_CALLBACK(message_received), pipeline);

  // Plant a pad probe to answer context queries
  GstElement *sink;
  sink = gst_bin_get_by_name(GST_BIN(vsink), "sink");
  if (sink) {
    GstPad *pad = gst_element_get_static_pad(sink, "sink");
    if (pad) {
      gulong id = gst_pad_add_probe(pad, GST_PAD_PROBE_TYPE_QUERY_DOWNSTREAM, pad_probe, pipeline, NULL);
      gst_object_unref(pad);
    }
    gst_object_unref(sink);
  }

Basically, in this case I just see blanking window (with different colors). The only difference in execution is that the pad_probe is called when using playbin3 but not when using udpsrc. This is the only difference that I can see adding some logs. I would like to understand why this callback is not called when using udpsrc and if I'm missimg something or using it wrong.

I'm facing the same issues both using gstreamer-1.14.4 and 1.16.2 version. Any hints are more than welcome.



from Gstreamer plugin cannot play video correctly on Android when using udpscr

No comments:

Post a Comment