Gstreamer video wall / grid

Related to my previous post: http://cogito.jonesmz.com/2016/02/merging-videos-using-gstreamer.html My brother also asked me to investigate if there was a way to make it easier to search through the videos. My idea was to display multiple videos side by side simultaneously, to make it easier to scan though what the camera captured.

So far, I've come up with this script

gst-launch-1.0 \

    videomixer \

        sink_1::zorder=1 sink_1::alpha=1.0 sink_1::ypos=0   sink_1::xpos=0    \

        sink_2::zorder=1 sink_2::alpha=1.0 sink_2::ypos=0   sink_2::xpos=630  \

        sink_3::zorder=1 sink_3::alpha=1.0 sink_3::ypos=0   sink_3::xpos=1260 \

        sink_4::zorder=1 sink_4::alpha=1.0 sink_4::ypos=496 sink_4::xpos=0    \

        sink_5::zorder=1 sink_5::alpha=1.0 sink_5::ypos=496 sink_5::xpos=630  \

        sink_0::zorder=1 sink_0::alpha=1.0 sink_0::ypos=496 sink_0::xpos=1260 \

        name=mix \

    uridecodebin uri=file:///path/to/video1.mkv ! videorate ! video/x-raw,framerate=[1/1,10/1] ! queue ! mix. \

    uridecodebin uri=file:///path/to/video2.mkv ! videorate ! video/x-raw,framerate=[1/1,10/1] ! queue ! mix. \

    uridecodebin uri=file:///path/to/video3.mkv ! videorate ! video/x-raw,framerate=[1/1,10/1] ! queue ! mix. \

    uridecodebin uri=file:///path/to/video4.mkv ! videorate ! video/x-raw,framerate=[1/1,10/1] ! queue ! mix. \

    uridecodebin uri=file:///path/to/video5.mkv ! videorate ! video/x-raw,framerate=[1/1,10/1] ! queue ! mix. \

    uridecodebin uri=file:///path/to/video6.mkv ! videorate ! video/x-raw,framerate=[1/1,10/1] ! queue ! mix. \

    mix. ! queue ! theoraenc ! progressreport name="Encoding Progress" ! matroskamux ! filesink location=/path/to/output/file.mkv

Note: The zorder and alpha attributes are only included for sake of example. Further, note that sink_0 is actually the last video attached, in this case video6. Personally I think this is a serious mistake on the part of the gstreamer developers, to use such confusing nomenclature, but it's their framework and they can implement it how they want to.

This pipeline does actually do the job, putting 6 videos in a 3x2 grid, and encoding the result to a single video file.

However, there are two nagging problems that I'm running into.

The first problem is that videomixer seems to be slower than molasses. It can take upward of ten minutes for this pipeline to finish merging six 10 second videos. I totally understand that the mixing process is cpu intensive, but it seems like it takes an excessive amount of time.

The second problem that I run into is that the resulting video, when I play it with VLC anyway, displays only the first frame. If I drag the scrollbar, I do see the other frames, but releasing the scrollbar makes the video revert right back to only displaying a single frame.

If anyone has any insights into how to improve this pipeline, I'd be very interested in any advice.