Motion Interpolation for Glitch Aesthetics using FFmpeg part 3

Below are a few examples of how you can use FFmpeg’s minterpolate to create artworks with a glitch aesthetic.

You can read about how I used it for an artwork in this blog post. You can also grab the source file for these videos here. Give it a try yourself!

mc_mode=obmc:me_mode=bidir:me=umh

ffmpeg -i cat_rainbow_original.mp4 -filter:v "setpts=62.5*PTS,minterpolate='fps=25:mb_size=16:search_param=400:vsbmc=0:scd=none:mc_mode=obmc:me_mode=bidir:me=umh'" 009_mc_mode=obmc_me_mode=bidir_me=umh.mp4

mc_mode=aobmc:me_mode=bidir:me=esa

ffmpeg -i cat_rainbow_original.mp4 -filter:v "setpts=62.5*PTS,minterpolate='fps=25:mb_size=16:search_param=400:vsbmc=0:scd=none:mc_mode=aobmc:me_mode=bidir:me=esa'" 010_mc_mode=aobmc_me_mode=bidir_me=esa.mp4

mc_mode=aobmc:me_mode=bidir:me=tss

ffmpeg -i cat_rainbow_original.mp4 -filter:v "setpts=62.5*PTS,minterpolate='fps=25:mb_size=16:search_param=400:vsbmc=0:scd=none:mc_mode=aobmc:me_mode=bidir:me=tss'" 011_mc_mode=aobmc_me_mode=bidir_me=tss.mp4

mc_mode=aobmc:me_mode=bidir:me=tdls

ffmpeg -i cat_rainbow_original.mp4 -filter:v "setpts=62.5*PTS,minterpolate='fps=25:mb_size=16:search_param=400:vsbmc=0:scd=none:mc_mode=aobmc:me_mode=bidir:me=tdls'" 012_mc_mode=aobmc_me_mode=bidir_me=tdls.mp4

This blog post is part of a series. Click the links below to see more examples of FFmpeg’s motion interpolation:

Motion Interpolation for Glitch Aesthetics using FFmpeg part 2

Below are a few examples of how you can use FFmpeg’s minterpolate to create artworks with a glitch aesthetic.

You can read about how I used it for an artwork in this blog post. You can also grab the source file for these videos here. Give it a try yourself!

mc_mode=obmc:me_mode=bidir:me=fss

ffmpeg -i cat_rainbow_original.mp4 -filter:v "setpts=62.5*PTS,minterpolate='fps=25:mb_size=16:search_param=400:vsbmc=0:scd=none:mc_mode=obmc:me_mode=bidir:me=fss'" 005_mc_mode=obmc_me_mode=bidir_me=fss.mp4

mc_mode=obmc:me_mode=bidir:me=ds

ffmpeg -i cat_rainbow_original.mp4 -filter:v "setpts=62.5*PTS,minterpolate='fps=25:mb_size=16:search_param=400:vsbmc=0:scd=none:mc_mode=obmc:me_mode=bidir:me=ds'" 006_mc_mode=obmc_me_mode=bidir_me=ds.mp4

mc_mode=obmc:me_mode=bidir:me=hexbs

ffmpeg -i cat_rainbow_original.mp4 -filter:v "setpts=62.5*PTS,minterpolate='fps=25:mb_size=16:search_param=400:vsbmc=0:scd=none:mc_mode=obmc:me_mode=bidir:me=hexbs'" 007_mc_mode=obmc_me_mode=bidir_me=hexbs.mp4

mc_mode=obmc:me_mode=bidir:me=epzs

ffmpeg -i cat_rainbow_original.mp4 -filter:v "setpts=62.5*PTS,minterpolate='fps=25:mb_size=16:search_param=400:vsbmc=0:scd=none:mc_mode=obmc:me_mode=bidir:me=epzs'" 008_mc_mode=obmc_me_mode=bidir_me=epzs.mp4

This blog post is part of a series. Click the links below to see more examples of FFmpeg’s motion interpolation:

Motion Interpolation for Glitch Aesthetics using FFmpeg part 1

Below are a few examples of how you can use FFmpeg’s minterpolate to create artworks with a glitch aesthetic.

You can read about how I used it for an artwork in this blog post. You can also grab the source file for these videos here. Give it a try yourself!

mc_mode=obmc:me_mode=bidir:me=esa

ffmpeg -i cat_rainbow_original.mp4 -filter:v "setpts=62.5*PTS,minterpolate='fps=25:mb_size=16:search_param=400:vsbmc=0:scd=none:mc_mode=obmc:me_mode=bidir:me=esa'" 001_mc_mode=obmc_me_mode=bidir_me=esa.mp4

mc_mode=obmc:me_mode=bidir:me=tss

ffmpeg -i cat_rainbow_original.mp4 -filter:v "setpts=62.5*PTS,minterpolate='fps=25:mb_size=16:search_param=400:vsbmc=0:scd=none:mc_mode=obmc:me_mode=bidir:me=tss'" 002_mc_mode=obmc_me_mode=bidir_me=tss.mp4

mc_mode=obmc:me_mode=bidir:me=tdls

ffmpeg -i cat_rainbow_original.mp4 -filter:v "setpts=62.5*PTS,minterpolate='fps=25:mb_size=16:search_param=400:vsbmc=0:scd=none:mc_mode=obmc:me_mode=bidir:me=tdls'"003_mc_mode=obmc_me_mode=bidir_me=tdls.mp4

mc_mode=obmc:me_mode=bidir:me=ntss

ffmpeg -i cat_rainbow_original.mp4 -filter:v "setpts=62.5*PTS,minterpolate='fps=25:mb_size=16:search_param=400:vsbmc=0:scd=none:mc_mode=obmc:me_mode=bidir:me=ntss'"004_mc_mode=obmc_me_mode=bidir_me=ntss.mp4

This blog post is part of a series. Click the links below to see more examples of FFmpeg’s motion interpolation:

Motion Interpolation for Glitch Aesthetics using FFmpeg part 0

As you may have seen in this blog post I made use of FFmpeg’s minterpolate motion interpolation options to make all of the faces morph. There’s quite a few options for minterpolate and many different combinations of options that can be used. i had to consult Wikipedia to figure out exactly what the different motion estimation algorithms were but even with that information I couldn’t visualise how it would change the output. To add to this how I’m using minterpolate isn’t a typical use case.

To make things easier for those wishing to use FFmpeg’s minterpolate to create glitch aesthetics I have compiled 36 videos each showing a different combination of processing options. The source video can be seen below and features two of my favourite things: cats (obtained from here) and rainbows.

I’ve slowed it down so that you can see exactly what’s in the video, but the original can be downloaded here.

The base script used for each video is:

ffmpeg -i cat_rainbow_original.mp4 -filter:v "setpts=62.5*PTS,minterpolate='fps=25:mb_size=16:search_param=400:vsbmc=0:scd=none:

In part two of March’s Development Update I explained why I set scd to none and search_param to 400. I could have of course documented what happens when all of the processing options are changed but that would result in me having to make hundreds of videos! The options that were changed were the mc_mode (motion compensation mode), me_mode (motion estimation mode), and me (motion estimation algorithm).

Test conditions

These videos were created using FFmpeg 7:4.1.4-1build2, installed from the Ubuntu repositories, on a Dell XPS 15 (2017 edition) with 16GB memory, a i7 processor and an Nvidia GeForce GTK 1050 graphics card, all running on Ubuntu 19.10 using proprietary drivers.

I don’t have a Windows or Mac machine, and haven’t used other Linux distributions so can’t test these scripts in those conditions. If there’s any problems with getting FFmpeg on your machine it’s best that you contact the developers for assistance.

Observations

My first observation is that the esa me_mode takes frikkin ages to complete! Each video using this me_mode took about four hours to process. I did consider killing the script but for completeness I let it run.

Using bilat me_mode produces the most chaotic results by far. Just compare 026_mc_mode=obmc_me_mode=bilat_me=epzs.mp4 to 008_mc_mode=obmc_me_mode=bidir_me=epzs.mp4 and you’ll see what I mean.

For a video of this length nearly all of the scripts (except for those using esa) took between 30 seconds and 1 minute to complete, and that’s on machines with and without a GPU. This is good news if you don’t want to have to carry around a powerhouse laptop all the time.

All of this reminds me a bit of datamoshing. It’s more predictable and controllable, but the noise and melty movement it creates, especially some of the ones using bilat me_mode, remind me of the bloom effect in datamoshing. This could be down to the source material, and I’d be interested to see experiments involving datamoshed videos.

Let’s a go!

With that all said let’s jump into sharing the results. As there’s 36 videos I’ll be splitting it over nine blog posts over nine days, with the last being posted on 28th March 2020. Each will contain the script I used as well as the output video. Links to each part can be found below:

Seamlessly loop Wave Modifier in Blender

Seamless animation

For the Improviz gifs one of the requirements that Rumblesan set is that the gifs loop seamlessly. That is, one would not be able to tell where the gifs beings and ends. In Blender making an animation seamless is pretty easy. There’s lots of examples out there but for completion here’s my simple take on it.

With the default cube selected press I and then press on Location. This inserts a keyframe for the location (this menu can also be accessed in Object > Animation > Insert Keyframe). On the Timeline at the bottom move the animation 20 frames. Then, move the cube to somewhere else.

Now press I to insert a keyframe for the location. Ta da! You now have an animation! To make it loop we need to repeat the first keyframe. On the Timeline go forward another 20 frames (so you’re now on frame 40). In the Timeline select the first keyframe. Press Shift + D to duplicate it and then move it to frame 40.

Set the end of your animation to be frame 40. Now when you press play (space bar) the animation loops seamlessly!

As an aside if you’re interested in animation check out Eadweard Muybridge. And if you’re into Pure Data check out this tutorial I made in 2017.

Seamlessly loop Wave Modifier

So, that’s one easy way to make a seamless looping animation. However, Rumblesan was more interested in are gifs that warp and morph. This is one example he sent me.

via GIPHY

In Blender one really useful modifier for making these animations is the Wave modifier. In fact, looking through all of the gifs in 2020 by that artist (Vince McKelvie) it looks like he makes extensive use of this modifier. I love how simple it is to get distorted objects without much effort.

The one thing I’ve always found difficult is making the looping of the waves seamless. I haven’t seen many tutorials on achieving this, and those that I have found rely a bit on guesswork, which isn’t ideal. So, I set out to understand this modifier. After a lot of trial and error and “maths” I finally consulted the documentation and started to figure it out! The documentation on this modifier is quite good but here’s my alternative explanation which may help those who think in a similar way to me.

To get your wave lasting a specific duration, first you need to know how long you want your animation to last. For this example I set mine to 50 frames.

You then need to decide on the Width of the waves. The smaller the number the more ripples you’ll have on your object. This value is relative to the object. So, if you set it to 0.10 you’ll have 10 ripples through your object. If you set it to 1 you’ll have one ripple. I’ve set mine to 0.25.

For the Speed you need to do a bit of maths. Copy the value of Width (0.25) and in the Speed argument enter: (0.25*2)/50. Replace 0.25 with whatever value you set for Width and 50 with however long your wave animation lasts before it loops. Another way to represent this would be:

Speed = ($width*2)/$animationlength

The animation loops however the waves don’t affect the whole object. This is because we need to add a negative offset so that the wave starts before the animation is visible. This is where we need more maths! Enter this into the Offset value:

((1/0.25)*50)*-1

The first part, 1/025, is to work out how many times we’d need to repeat the Width before the whole object has ripples throughout it. We multiply by 50 as that is the animation duration. Then, we multiply by -1 to get the inverse, which because the offset. Another way to represent this would be:

Offset = ((1/$width)*$animationlength)*-1

And now the whole object has waves through it and loops seamlessly!

Ta da!

Since I originally made the gifs I have found that there are alternative methods for achieving a wavy object which rely on the displacement node and Displacement modifier or the Lattice or Cast modifier. These solutions have much more documentation but I’m glad I spent the time figuring out the Wave modifier.

The Barber Collective Exhibition

The Barber Collective re-interpret the collection through interdisciplinary art, music and performance practice. Collaborating with artists, designers and musicians, the group of 16 – 21 year olds discuss, re-imagine and create. Enjoy this exhibition of work by the first cohort.

Bus Tops video

If you weren’t able to make it out to the streets of London in February to catch the Bus Tops here’s a video featuring a selection of the works:

You can see one of my pieces show up at 2:08. There’s some more videos from the project on al4ie’s YouTube channel

Hello Glitch

Using the same svg glitching techniques that I’ve come to love, today, with the help of Twitter folk, I created this rather short animation that compliments my new deviantART ID

My initial aim was glitch the video file itself but then I figured it’d be quite different to glitch each frame individually. To do this I created the original text in Inkscape and then created 52 copies of it. I then opened up the file in a text editor and began messing around with the numbers!

After the 17th or so version of the file it became hard to try and randomly replace numbers, so I turned to Twitter for some help.

I asked people to give me two random numbers, each between 0 and 99. I’d then modify an image using only those two numbers. Within minutes of asking I already was given quite a lot of numbers, so I began work on glitching. In the end I sporadically used the numbers that were given on each image. All of the results can be seen below:

Hello Glitch (by hellocatfood)

All but two of the frames from the animation in no particular order

The amount of variation you can get by just replacing a few numbers or changing a number from a negative to a positive is quite amazing.

For the sound I ran an mp3 file through MPeg Fucker, which is a nifty little script for warping sounds.

hEll



hEll, originally uploaded by hellocatfood.

Something from a little animation I’m working on