Motion Interpolation for Glitch Aesthetics using FFmpeg part 3

Below are a few examples of how you can use FFmpeg’s minterpolate to create artworks with a glitch aesthetic.

You can read about how I used it for an artwork in this blog post. You can also grab the source file for these videos here. Give it a try yourself!

mc_mode=obmc:me_mode=bidir:me=umh

ffmpeg -i cat_rainbow_original.mp4 -filter:v "setpts=62.5*PTS,minterpolate='fps=25:mb_size=16:search_param=400:vsbmc=0:scd=none:mc_mode=obmc:me_mode=bidir:me=umh'" 009_mc_mode=obmc_me_mode=bidir_me=umh.mp4

mc_mode=aobmc:me_mode=bidir:me=esa

ffmpeg -i cat_rainbow_original.mp4 -filter:v "setpts=62.5*PTS,minterpolate='fps=25:mb_size=16:search_param=400:vsbmc=0:scd=none:mc_mode=aobmc:me_mode=bidir:me=esa'" 010_mc_mode=aobmc_me_mode=bidir_me=esa.mp4

mc_mode=aobmc:me_mode=bidir:me=tss

ffmpeg -i cat_rainbow_original.mp4 -filter:v "setpts=62.5*PTS,minterpolate='fps=25:mb_size=16:search_param=400:vsbmc=0:scd=none:mc_mode=aobmc:me_mode=bidir:me=tss'" 011_mc_mode=aobmc_me_mode=bidir_me=tss.mp4

mc_mode=aobmc:me_mode=bidir:me=tdls

ffmpeg -i cat_rainbow_original.mp4 -filter:v "setpts=62.5*PTS,minterpolate='fps=25:mb_size=16:search_param=400:vsbmc=0:scd=none:mc_mode=aobmc:me_mode=bidir:me=tdls'" 012_mc_mode=aobmc_me_mode=bidir_me=tdls.mp4

This blog post is part of a series. Click the links below to see more examples of FFmpeg’s motion interpolation:

Motion Interpolation for Glitch Aesthetics using FFmpeg part 2

Below are a few examples of how you can use FFmpeg’s minterpolate to create artworks with a glitch aesthetic.

You can read about how I used it for an artwork in this blog post. You can also grab the source file for these videos here. Give it a try yourself!

mc_mode=obmc:me_mode=bidir:me=fss

ffmpeg -i cat_rainbow_original.mp4 -filter:v "setpts=62.5*PTS,minterpolate='fps=25:mb_size=16:search_param=400:vsbmc=0:scd=none:mc_mode=obmc:me_mode=bidir:me=fss'" 005_mc_mode=obmc_me_mode=bidir_me=fss.mp4

mc_mode=obmc:me_mode=bidir:me=ds

ffmpeg -i cat_rainbow_original.mp4 -filter:v "setpts=62.5*PTS,minterpolate='fps=25:mb_size=16:search_param=400:vsbmc=0:scd=none:mc_mode=obmc:me_mode=bidir:me=ds'" 006_mc_mode=obmc_me_mode=bidir_me=ds.mp4

mc_mode=obmc:me_mode=bidir:me=hexbs

ffmpeg -i cat_rainbow_original.mp4 -filter:v "setpts=62.5*PTS,minterpolate='fps=25:mb_size=16:search_param=400:vsbmc=0:scd=none:mc_mode=obmc:me_mode=bidir:me=hexbs'" 007_mc_mode=obmc_me_mode=bidir_me=hexbs.mp4

mc_mode=obmc:me_mode=bidir:me=epzs

ffmpeg -i cat_rainbow_original.mp4 -filter:v "setpts=62.5*PTS,minterpolate='fps=25:mb_size=16:search_param=400:vsbmc=0:scd=none:mc_mode=obmc:me_mode=bidir:me=epzs'" 008_mc_mode=obmc_me_mode=bidir_me=epzs.mp4

This blog post is part of a series. Click the links below to see more examples of FFmpeg’s motion interpolation:

Motion Interpolation for Glitch Aesthetics using FFmpeg part 1

Below are a few examples of how you can use FFmpeg’s minterpolate to create artworks with a glitch aesthetic.

You can read about how I used it for an artwork in this blog post. You can also grab the source file for these videos here. Give it a try yourself!

mc_mode=obmc:me_mode=bidir:me=esa

ffmpeg -i cat_rainbow_original.mp4 -filter:v "setpts=62.5*PTS,minterpolate='fps=25:mb_size=16:search_param=400:vsbmc=0:scd=none:mc_mode=obmc:me_mode=bidir:me=esa'" 001_mc_mode=obmc_me_mode=bidir_me=esa.mp4

mc_mode=obmc:me_mode=bidir:me=tss

ffmpeg -i cat_rainbow_original.mp4 -filter:v "setpts=62.5*PTS,minterpolate='fps=25:mb_size=16:search_param=400:vsbmc=0:scd=none:mc_mode=obmc:me_mode=bidir:me=tss'" 002_mc_mode=obmc_me_mode=bidir_me=tss.mp4

mc_mode=obmc:me_mode=bidir:me=tdls

ffmpeg -i cat_rainbow_original.mp4 -filter:v "setpts=62.5*PTS,minterpolate='fps=25:mb_size=16:search_param=400:vsbmc=0:scd=none:mc_mode=obmc:me_mode=bidir:me=tdls'"003_mc_mode=obmc_me_mode=bidir_me=tdls.mp4

mc_mode=obmc:me_mode=bidir:me=ntss

ffmpeg -i cat_rainbow_original.mp4 -filter:v "setpts=62.5*PTS,minterpolate='fps=25:mb_size=16:search_param=400:vsbmc=0:scd=none:mc_mode=obmc:me_mode=bidir:me=ntss'"004_mc_mode=obmc_me_mode=bidir_me=ntss.mp4

This blog post is part of a series. Click the links below to see more examples of FFmpeg’s motion interpolation:

Motion Interpolation for Glitch Aesthetics using FFmpeg part 0

As you may have seen in this blog post I made use of FFmpeg’s minterpolate motion interpolation options to make all of the faces morph. There’s quite a few options for minterpolate and many different combinations of options that can be used. i had to consult Wikipedia to figure out exactly what the different motion estimation algorithms were but even with that information I couldn’t visualise how it would change the output. To add to this how I’m using minterpolate isn’t a typical use case.

To make things easier for those wishing to use FFmpeg’s minterpolate to create glitch aesthetics I have compiled 36 videos each showing a different combination of processing options. The source video can be seen below and features two of my favourite things: cats (obtained from here) and rainbows.

I’ve slowed it down so that you can see exactly what’s in the video, but the original can be downloaded here.

The base script used for each video is:

ffmpeg -i cat_rainbow_original.mp4 -filter:v "setpts=62.5*PTS,minterpolate='fps=25:mb_size=16:search_param=400:vsbmc=0:scd=none:

In part two of March’s Development Update I explained why I set scd to none and search_param to 400. I could have of course documented what happens when all of the processing options are changed but that would result in me having to make hundreds of videos! The options that were changed were the mc_mode (motion compensation mode), me_mode (motion estimation mode), and me (motion estimation algorithm).

Test conditions

These videos were created using FFmpeg 7:4.1.4-1build2, installed from the Ubuntu repositories, on a Dell XPS 15 (2017 edition) with 16GB memory, a i7 processor and an Nvidia GeForce GTK 1050 graphics card, all running on Ubuntu 19.10 using proprietary drivers.

I don’t have a Windows or Mac machine, and haven’t used other Linux distributions so can’t test these scripts in those conditions. If there’s any problems with getting FFmpeg on your machine it’s best that you contact the developers for assistance.

Observations

My first observation is that the esa me_mode takes frikkin ages to complete! Each video using this me_mode took about four hours to process. I did consider killing the script but for completeness I let it run.

Using bilat me_mode produces the most chaotic results by far. Just compare 026_mc_mode=obmc_me_mode=bilat_me=epzs.mp4 to 008_mc_mode=obmc_me_mode=bidir_me=epzs.mp4 and you’ll see what I mean.

For a video of this length nearly all of the scripts (except for those using esa) took between 30 seconds and 1 minute to complete, and that’s on machines with and without a GPU. This is good news if you don’t want to have to carry around a powerhouse laptop all the time.

All of this reminds me a bit of datamoshing. It’s more predictable and controllable, but the noise and melty movement it creates, especially some of the ones using bilat me_mode, remind me of the bloom effect in datamoshing. This could be down to the source material, and I’d be interested to see experiments involving datamoshed videos.

Let’s a go!

With that all said let’s jump into sharing the results. As there’s 36 videos I’ll be splitting it over nine blog posts over nine days, with the last being posted on 28th March 2020. Each will contain the script I used as well as the output video. Links to each part can be found below:

Gamerz Festival

DIGITAL | ALTER OUVERTURE & VERNISSAGE

Vernissage, MERCREDI 13 NOVEMBRE à partir de 14H
Fondation Vasarely en présence des artistes
15e Édition du Festival des arts multimedia GAMERZ
http://www.festival-gamerz.com/gamerz15/

EXPOSITIONS | du 13 au 24 novembre 2019

FONDATION VASARELY :
Avec les ensembles monographiques :
« DEMAIN LES ROBOTS » FRANCE CADET
« L’INTRIGUE » FABRICE MÉTAIS
« MEME » OLIVIER MORVAN
« 3615 LOVE » PAMAL_Group
avec : JACQUES-ELIE CHABERT, EDUARDO KAC,
MARIE MOLINS, CAMILLE PHILIBERT
et les œuvres de :
JULIUS VON BISMARCK
JON RAFMAN
ANTONIO ROBERTS

OFFICE DU TOURISME :
Avec les œuvres de :
MANUEL BRAUN & ANTONIN FOURNEAU
ROBIN MORETTI & LESLIE ASTIER
ROBIN MORETTI & THÉO GOEDERT

Le Festival international des Arts Multimédia GAMERZ revient pour sa 15e édition cet automne du 13 au 24 novembre 2019 dans la ville d’Aix-en-Provence, après une pré-ouverture inédite et vibrante à Marseille. Le Festival GAMERZ déploie une programmation originale, internationale et résolument expérimentale en invitant de nombreux artistes à investir différents lieux de création contemporaine. Cette année, le festival interroge notre condition contemporaine en proposant la notion de « Digital Alter » comme axe de recherche et chantier artistique.

https://www.facebook.com/events/415769632671647/
https://www.fondationvasarely.org/expositions-et-actualites/les-expositions/

Radiophonic

Edited arts presents: Radiophonic
A tribute to the BBC Radiophonic Workshop on its 60th anniversary

A day and night event – a mini-festival – at Down Lane Studios, a brand new multi-functional 400 capacity ex-industrial space in Tottenham.

Live musicians, DJs, talks, workshops, film screenings and multi-disciplinary artworks.

Late at Tate Britain: Echoes

A night exploring myths and morality, inspired by Margerite Humeau’s Echoes

Late at Tate Britain April explores the Ancient Egyptians’s relationship with mortality and its parallels with contemporary society, inspired by Margeritue Humeau. Ancient Egyptians looked at preserving life trough spirituality. In current times, do we try to achieve this through digital formats.

The series will kick off on 6 April with an evening exploring the current Art Now installation by Marguerite Humeau. The work entitled Echoes is conceived as a confrontation between life and death, with the gallery transformed into part temple, part laboratory for the industrial production of an elixir for eternal life.

Glitch Art Workshops

Glitch art is a form of art that is created by introducing digital or analog errors to media.

Join MakoEducation and Near Now Fellow Antonio Roberts in this glitch art workshop which teaches participants what glitch art is and how to create their own artwork.

Attendees will learn how to create glitch art images, videos and GIFs. We’ll be encouraging artists of the future to experiment with new tools and techniques.

Attendees learn a range of digital skills in the areas of photography, image editing and video production.

No previous experience is necessary to take part in this workshop.

PLEASE NOTE THIS EVENT IS AIMED AT CHILDREN AGED 8-13. ADULTS PERMITTED ONLY TO ACCOMPANY CHILDREN. ALL CHILDREN MUST BE ACCOMPANIED BY AN ADULT.

Granular Colloquium

Utilising a range of formats from audio-visual performance to talks, this event is an experiential investigation of noise as a granular entity. State changes are a central theme. Processes of disintegration and/or reintegration of material elements at a granular level are explored, both as the mode of transference between states (whether physical or digital) and the means by which a thing starts or ceases to be.

The colloquium will take place from 10 am – 5pm at The University of Greenwich, Stockwell Street Building and is held in association with the exhibition Granular: The Material Properties of Noise. The event will be followed by a private viewing of the exhibition.

Chaired by Dr. Stephen Kennedy, University of Greenwich, Department of Creative Professions and Digital Arts
(author: Chaos Media: A Sonic Economy of Digital Space – Bloomsbury 2015)
Keynote: Greg Hainge , University of Queensland, Associate Professor, School of Languages and Cultures
(author: Noise Matters: Towards an Ontology of Noise – Bloomsbury 2013)
Contributors include Russell Duke, Jane Grant, Antonio Roberts, Dr David Ryan, Charles Danby and Rob Smith.

Tickets cost £10 (£5 for students) and are available via this link to our Eventbrite page. Tickets include free access to the performance of “Recitativo” on Friday 26th January.

Granular: The Material Properties of Noise, 16th January – 3rd February

From 16th January – 3rd February my 2016 piece Transformative Use will be on show at University of Greenwich as part of Granular: The Material Properties of Noise.

Granular noise is explored as a condition of material transfer in this exhibition. A central concern across the works on display is the material state change that occurs within the processes of mediation. Here, disintegration and/or reintegration of elements at a granular level is encountered as a mode of transference between states, whether physical or digital, and as a phase at which a thing starts or ceases to be.

Exhibiting artists include: Jim Hobbs, Benjamin McDonnell, Antonio Roberts, David Ryan, Audrey Samson and Rob Smith.

The exhibition features my work Transformative Use, which was originally commissioned by Hannah Pierce for the Common Property exhibition in 2016. It’s the first time it’s been exhibited since then. If you want to see some work-in-progress installation shots check out my all new arty Instagram account.

Alongside the exhibition is the Granular Colloquium, taking place on 27th January:

Utilising a range of formats from audio-visual performance to talks, this event is an experiential investigation of noise as a granular entity. State changes are a central theme. Processes of disintegration and/or reintegration of material elements at a granular level are explored, both as the mode of transference between states (whether physical or digital) and the means by which a thing starts or ceases to be.

I’ll be at that, talking a bit about glitch and its relation to copyright, as well as regular ol’ copyright. Tickets are £10.

If you didn’t get the chance to see Transformative Use in 2016 now is a great time to see a new and updated configuration of it.