Monday, February 8, 2016

Merging videos using gstreamer

My brother came to me the other day asking if I knew of a way to merge multiple small (approximately 30 seconds each) video files into a single, longer, video file. He works for a retail business and wanted to review some security footage more easily.

The GStreamer framework, as of version 1.6, now has a plugin that makes doing this operation significantly easier than ever before. The concat plugin simply takes 1 to N number of inputs, and activates those inputs one at a time, playing each entirely, filtering the EOS event from all but the last element, and then moving to the next. In this way, we can make other elements in the pipeline treat these multiple sources of data as a single, continuous, stream.

The ultra-simplified explanation is that we're simply playing each small video one after another with no pause in between, and then re-encoding the resulting video as a single file.

The problem that I was running into is two fold. First, the some aspect of the contact element, when running from the commandline using the gst-launch-1.0 debugging / prototyping program, becomes very hostile to handling more than, for example, 10 files at a time. I was able to consistently use a pipeline that looks like this

gst-launch-1.0 concat name=c \
    uridecodebin uri=file:///path/to/video1.asf ! queue ! c. \
    uridecodebin uri=file:///path/to/video2.asf ! queue ! c. \
    uridecodebin uri=file:///path/to/video3.asf ! queue ! c. \
    uridecodebin uri=file:///path/to/video4.asf ! queue ! c. \
    uridecodebin uri=file:///path/to/video5.asf ! queue ! c. \
    c. ! videoconvert ! theoraenc ! progressreport name="Encoding\ Progress" ! matroskamux ! filesink location=/path/to/output/file.mkv ;
To merge a handful of files at a time, but trying to do more than roughly 10 made gst-launch throw up errors very consistently

My second problem, which gets drastically compounded by the first problem, is that my brother handed me a thumb drive with over 5 thousand video files on it. Personally, I have no interest in sitting there typing out each file name one at a time, and to the best of my, admittedly limited) knowledge, gstreamer simply doesn't have an element that opens each file that matches some pattern and attaches them to a sink in a way that would be useful for this situation.

So, I need to write a meta program that generates a program that calls gst-launch with only a few video files at a time. I figured that if I could merge them 5 at a time, I could merge the resulting videos 5 at a time, and repeat that pattern until there were only around a dozen big video files, at which point he can search through them manually on his own.

So here's what I've come up with so far. It's a bit of a big-huge-hack, but at least it'll let me get the job done and move onto other projects.
mkdir -p encode_results
echo "runfunction() {" >> $OUTPUTSCRIPT
echo "    until \$@" >> $OUTPUTSCRIPT
echo "    do" >> $OUTPUTSCRIPT
echo "        :" >> $OUTPUTSCRIPT
echo "    done" >> $OUTPUTSCRIPT
echo "}" >> $OUTPUTSCRIPT
echo "" >> $OUTPUTSCRIPT
for f in `ls *.ASF | sort -g` ;
    if [ $COUNTER -eq 0 ] ;
        echo "runfunction gst-launch-1.0  --no-fault --gst-debug-disable concat name=c \\" >> $OUTPUTSCRIPT
    echo "    uridecodebin uri=file://$VIDEOFOLDER/$f ! videorate ! video/x-raw,framerate=[1/1,10/1] ! queue ! c. \\" >> $OUTPUTSCRIPT
    if [ $COUNTER -eq $NUMBER_PER_SWEEP ] ;
        echo "    c. ! videoconvert ! theoraenc ! progressreport name=\"Encoding\ Progress\" ! matroskamux ! filesink location=$VIDEOFOLDER/encode_results/encode_result_${FIRST}_${f}.mkv &" >> $OUTPUTSCRIPT
        echo "" >> $OUTPUTSCRIPT
if [ $COUNTER -ne 0 ] ;
    echo "    c. ! videoconvert ! theoraenc ! progressreport name=\"Encoding\ Progress\" ! matroskamux ! filesink location=$VIDEOFOLDER/encode_results/encode_result_${FIRST}_${f}.mkv &" >> $OUTPUTSCRIPT
    echo "" >> $OUTPUTSCRIPT
You should be able to simply copy it into a bash prompt after changing the filenames at the top.

It'll generate a new script. That new script will all the encoding jobs in parallel. If any encoding job fails to complete successfully, then the "runfunction" will automatically start it again.

1 comment:

  1. M88 Casino Review - Bonuses, Software & Games
    M88 Casino offers a variety of online slots and games from Microgaming, including a massive collection of live william hill dealer games like Games: 550+ Slots, 100+ Table GamesBonus: Welcome Bonus up to m88 ทางเข้า ₦2,000 1xbet + 50 Free Spins