I Am Sitting in a Room Revisited

My remake of the Alvin Lucier artwork I Am Sitting in a Room was released 10 years ago today. To mark this occasion I want to revisit how I made it and troubles I had remaking it for 2019.

In November 2019 I Am Sitting in a Room was exhibited at Gamerz Festival in Aix-en-Provence in France. It was a pretty nice surprise to be invited to exhibit that piece, especially as it was made in 2010 and hasn’t been exhibited since aroudn 2015.

The festival’s organiser, Quentin, asked if I could remake the film in HD resolution, which is 1920×1080/16:9 aspect ratio. The original video was made in the very odd resolution of 1440×1152, which is 4:3 aspect ratio. In any normal situation using filmed footage, this could be an impossible task. If you wanted to convert 4:3 to 16:9 you’d either have to stretch the footage or crop it and lose information. See these examples below:

Crop to fit

Stretch to fit

For I Am Sitting in a Room in theory this isn’t a problem as the piece is black text against a white background. So, I would just need to make the background larger and center the text.

Before going into the problems I faced remaking the piece in 16:9 it’s worth going over how it was made. Back in 2010 my programming skills were, to be blunt, crap. Making this piece required me to learn a lot about Linux, loops, automation and bash programming. Also back in 2010 I wasn’t that good at documenting my processes and so looking back at my source files some of them were not in a good state or were just missing. So, part of the following is pieced together from memory and what remains on my hard drives.

In 2009 I wrote about “glitching” SVG files and became really interested in that as a format to work with. Even today I like working with SVGs as they’re highly editable and can be used in many ways (vinyl cutting, plotting, laser printing, screen printing, web design etc).

If you’ve ever done any glitching via the command line you’ll know that you can utilise sed to automate the glitching of files. sed didn’t quite work on SVGs as it would always destroy them and so I asked Garry Bulmer to write a script to glitch SVG files. It worked great and left the SVGs in an editable state.

At some point later it was brought to my attention that font glyphs are formed in the same way as vector files in that the glyphs are made up of paths and nodes, hence why they are infinitely scalable. SVG fonts was/is a specification for, well, representing fonts as an SVG file. So, if I could convert a font to SVG then in theory I could glitch it!

First I needed a font to work with. The very popular Ubuntu font wasn’t out at that point and so I opted to use Liberation Sans. It’s a free font and had a lot of glyphs.

Using FontForge I converted the font to an SVG font (File > Generate Fonts). I then used the glitch script from Garry Bulmer in a way like so:

#!/bin/bash
no=1
while [ $no -le 1001 ]
do
echo $no
./glitch 0.1 < font.svg > font.svg
cp font dataface_$no.svg
no=`expr $no + 1`
done

Then I had a folder of 1002 SVG fonts. I needed to turn them back to ttf files and so used the follwing font forge scrpt to convert them back to ttfs

#!/usr/local/bin/fontforge
Open($1)
Generate($1:r + ".ttf")

And ran it over the whole folder with this cript

#!/bin/bash
no=1
while [ $no -le 1001 ]
do
echo $no
fontforge -script convert_font_fontforge.pe dataface_$no.svg
no=`expr $no + 1`
done

Finally, I generated the frames used for the video by creating an SVG file with the text in that uses the Liberation Sans font. For each frame I swapped the font file with the next one which was a bit glitchier:

#!/bin/sh
no=1
while [ $no -le 1001 ]
do
echo $no
sudo rm /home/hellocatfood/.fonts/*.ttf
cp /home/hellocatfood/Desktop/dataface_$no.ttf /home/hellocatfood/.fonts/
convert /home/hellocatfood/Desktop/glitch.svg /home/hellocatfood/Desktop/glitch_$no.jpg
no=`expr $no + 1`
done

ta da! Not a very elegant soluion but it was 2010. And it worked! The result you already saw at the beginning of this blog post. So, to make a HD version of it, in theory all I need to do was create an SVG file that was 16:9/1920×1080 and repeat this process. Here’s the result of that process:

Yikes! Not at all like the original.

You may be asking why I don’t just convert the text to paths (Ctrl + Shift + C in Inkscape). That would do away with the need to generate 1002 font files. Here’s what that would have looked like:

However, it differs from the original in one unique way. Once the glitches start there’s a lot of random question marks that appear and float down he screen. I don’t know why this happen but I suspect that Imagemagick doesn’t know how to interpret the glitched glyphs and so produces and error.

There are also some frames which fail to render the font at all and in a couple of instances render a serif font!

The results in the original are not consistent, but the piece is an exploration of glitches and so whatever glitch was produced is whatever glitch I used. So there.

Therefore, any remake which didn’t look like the original wouldn’t be faithful to the original, and I simply didn’t want to exhibit a compromised video.

One promising solution which – TL;DR – didn’t work was to try and replicate the environment I made the original work in. That is, to try and replicate an Ubuntu computer from 2010. I booted up Ubuntu 10.10 on a virtual machine (using Virtual Box) and got to work.

Ubuntu 10.10

Imagemagick was failing to read the font files and so I used GIMP on the command line to read and convert the SVG to pngs.

gimp -n -i -b - <<EOF
(let* ( (file's (cadr (file-glob "sitting_in_a_room.svg" 1))) (filename "") (image 0) (layer 0) )
(while (pair? file's)
(set! image (car (gimp-file-load RUN-NONINTERACTIVE (car file's) (car file's))))
(set! layer (car (gimp-image-merge-visible-layers image CLIP-TO-IMAGE)))
(set! filename (string-append (substring (car file's) 0 (- (string-length (car file's)) 4)) ".png"))
(gimp-file-save RUN-NONINTERACTIVE image layer filename filename)
(gimp-image-delete image)
(set! file's (cdr file's))
)
(gimp-quit 0)
)
EOF

cp sitting_in_a_room.png images/glitch_$no.png

Which produced this result:

This method shows error characters but not the same question mark error characters that were in the original. No matter what I tried it just wasn’t working.

So, in the end unfortunately I admitted defeat and exhibited the original 4:3 video. It would be easy to blame the 2010 version of myself for not creating an air-gapped iso of Ubuntu or for not properly documenting my processes, but how was I to know that I would be revisiting the piece 10 years later! Heck, in 2010 HD resolution was still not widespread, so I was really just working with what I knew.

And really, this highlights the volatility of glitches.

OCR to Text to Speech

For the sixth video in the Design Yourself series the group worked with artist Erica Scourti. For the activity the participants used optical character recognition software (OCR) to generate poetry from their own handwriting and writing (leaflets, signage) found throughout the Barbican building.

The next stage in the workshop was going to be to take this extracted text and run it through a text to speech synthesizer, but unfortunately there wasn’t time to get to this stage.

One of the things I liked about the software they used was that it showed you the image of the text that it recognised and extracted, producing a kind of cut-and-paste poetry.

To make the sixth video I wanted to somehow utilize this OCR and text-to-speech process and make a video collage of words and synthesized speech. The challenge was finding a way to do this using only open source software. Finding open source OCR software that works on Linux is not a problem. After a while I discovered that Tesseract is the gold standard for OCR software and that most other software act as frontends or interfaces to it. Here’s a few examples:

However, they all output only the text, and not the image of the extracted text. I’m aware that my use case is quite specific so I don’t blame the developers for this.

Eventually I took to Twitter and Mastodon with my questions. _vade pointed to a bug report on Tesseract which showed that getting the coordinates of recognised words is possible in Tesseract. If I knew the coordinates of words then perhaps I could use that to extract the image of the word. However, doing it this way required using it’s C interface and learning C wasn’t feasible at the time.

After some further digging around Tesseract I found a bug report that makes reference to hOCR files:

hOCR is an open standard of data representation for formatted text obtained from optical character recognition (OCR). The definition encodes text, style, layout information, recognition confidence metrics and other information using Extensible Markup Language (XML) in the form of Hypertext Markup Language (HTML) or XHTML.

This file looked like it contained the coordinate data I needed, and obtaining such a file from Tesseract was as simple as running one command. The next task was finding a tool (or tools) to interpret hOCR files. Here’s a selection, which should really be added the the previous list to form a mega-list:

hocr-tools proved to be the most feature complete and stable. It runs on the command line, which opens it up for easy automation and combining with other programs. After reading the documentation I found a process for extracting the images of words and even making videos from each word/sentence with synthesized speech. Here’s how I did it:

Generate hOCR file

First I needed to generate a hOCR file using Tesseract. For the example I used the first page from the first chapter of No Logo by Naomi Klein.

tesseract book.jpg book hocr

This produced a file called book.hocr. If you look at the source code of the file you can see it contains the bounding box coordinates of each word and line.

Extract images

Using the hOCR file I can extract images of the lines

hocr-extract-images -P 10 book.hocr

This generates both pngs of each line and also a corresponding text file containing the text

Text to speech

Using eSpeak I can generate a wav file of a synthesized voice reading each line.

for file in *.txt ; do espeak -z -f $file -w ${file%.*}.wav ; done

Make video clips

Finally, I needed to combine the png image of the text with the wav file of the synthesized speech into a video

for i in $(seq -f "%03g" 1 83) ; do ffmpeg -loop 1 -i line-$i.png -i line-$i.wav -c:v libx264 -tune stillimage -vf scale="width=ceil(iw/2)*2:height=ceil(ih/2)*2" -pix_fmt yuv420p -shortest -fflags +shortest ${i%.*}.mp4 -y ; done

I used fflags because without it the video was always adding a couple of seconds of silence.

By the end of this I had a folder full of lots of files.

Voila!

The last part of this was to manually arrange the video clips into a video collage. I made this example video to demonstrate to the group what could be done.

Getting to this point took some time but with what I’ve learnt I can replicate this process quickly and simply. In the end the group decided no to use OCR to generate text and instead wrote something themselves. They did still use text-to-speech software and even filmed themselves miming to it. Here’s the finished video:

This was the last video I made for the Design Yourself project. I’ve written about techniques used to make older videos in past blog posts. Go read the Barbican website for more information on the project

Copy Paste photos

Photos of the Copy Paste exhibition currently taking place at Piksel in Bergen, Norway, and online in the Piksel Cyber Salon

Copy Paste

Copy Paste

Copy Paste

Copy Paste

Copy Paste

Copy Paste

Copy Paste

Full list of exhibiting artists inculde Carol Breen, Constant, LoVid, Lorna Mills, Matthew Plummer-Fernandez + Julien Deswaef, Duncan Poulton, Eric Schrijver, Peter Sunde.

Photos taken by Maite Cajaraville. More photos can be seen here.

If you have the opportunity to see the exhibition in person please do! It’s open until 21st June.

Copy Paste opening

Copy Paste opened at Piksel in Bergen on the evening of Friday 22nd May. Sadly myself and all of the exhibiting artists were unable to be there but fortunately they live streamed the whole thing.

After all of the uncertainty about whether Copy Paste would go ahead I’m really happy that situation in Bergen has been good enough for the exhibition to welcome visitors. It was sad to not to be in Bergen myself to see everything IRL but I’m thankful to Maite and Gisle, Directors of Piksel, for handling all of the logistics and installation of the works.

Copy Paste exists in two spaces. In the physical studio space visitors can find works by Carol Breen, Constant, Lorna Mills, Duncan Poulton, Eric Schrijver, and Peter Sunde.

Here’s a pixelated look at some of the artworks as captured by me in the UK from the livestream:

Copy Paste also exists as a virtual online exhibition in the Piksel Cyber Salon.

The space is built using Mozilla Hubs and works in your web browser (or VR headset if you have one). In the Cyber Salon you can find works by LoVid, Matthew Plummer-Fernandez + Julien Deswaef, Carol Breen, and Duncan Poulton.

Many thanks Malitzin Cortes for designing this space. You can visit it at any time and all of the live streamed events will also be streamed to there.

Events

Speaking of events check out these upcoming events happening as part of Copy Paste!

Curator’s Tour

24th, 31st, 7th, 14th, 21st June 13:00 – 14:00
Each Sunday at 13:00 – 14:00 CEST I’ll be giving a tour of the exhibition (remotely, obvs), talking a bit about each artwork and how they contribute to the exhibition and explore ideas around copying.

Live Coding Algorave Performance with Alex McLean and Antonio Roberts

29th May 23:00 – 00:00 CEST
On 29th May 23:00 – 00:00 myself and Alex McLean will be doing a live coding performance. Alex will be doing his usual patterns of sample based music and visually I’ll be mixing things up a bit.

Authors of the Future

6th June 18:00 – 20:00 CEST
An online presentation from Constant of Authors of the Future, with a focus on the Cinemas Sauvage license. This license shows the pitfalls and fun (im)possibility of coming to an agreement with a bunch of anarchist people who do not want to agree on a rule.

Internet Archaeology for Beginners

7th June 16:00 – 18:00 CEST
Join artist Duncan Poulton on 7th June 16:00 – 18:00 CEST for a virtual workshop which offers an introduction to techniques for mining and misusing the web for creative reuse. Attendees will visit the depths of the internet that search engines don’t want you to find, and learn to make their own digital collages from the materials they gather.

To book onto Duncan’s workshop and find out more about the other eents send an e-mail to piksel20(at)piksel(dot)no

Hope y’all enjoy the exhibitoin!

Copy Paste – 22nd May – 21st June 2020

I’m happy to announce that I am curating the exhibition Copy Paste, taking place at Piksel in Norway from 22nd May – 21st June.

If you are an artist, then you have no doubt copied the work of others. This copying can range from using pages from a magazine in collage, adopting the style of an artist, or simply being inspired by the work of an artist. This natural process of copying, taught to us at every stage of our artistic development, is burdened by a very complex and messy set of laws and social conventions which define and limit how we can use copying within our practices. These don’t take into account exceptions or nuances, and come from a historical world where artworks were scarce physical objects, and don’t translate well into a world where culture is abundant and can be accessed and copied at will.

Being an artist who copies is to be an artist working in this murky grey area of right and wrong.

For those artists, I have curated Copy Paste. This exhibition features the work of nine artists and art collectives who all incorporate copying as a core aspect of their work. Taking the form of a physical exhibition at Piksel in Norway, an online exhibition, and an event series, the exhibition aims to show that copying is natural.

Exhibiting artists: Carol Breen, Constant, LoVid, Lorna Mills, Matthew Plummer-Fernandez + Julien Deswaef, Duncan Poulton, Eric Schrijver, Peter Sunde.

There will be a few events happening throughout the exhibition, taking place online and IRL at Piksel’s studio in Norway. More details of that will follow here and on Piksel’s website. Thanks so much to Piksel for the invitation to curate this exhibition 🙂

Gifhouseparty – 16th May 2020

On 16th May 22:00 BST I’ll be doing a gif-enfused audiovisual performance for the Well Now WTF? exhibition.

In the 30 minute performance I’ll be live coding music and sounds for all of the gifs currently stuck at home! If you want to be at the party send me a gif of yourself dancing against a green screen or solid colour by 12th May and tune in to the performance over on Twitch 🙂

Seamless Looping Neon Trail in Blender

I’d like to return to the fifth video in the Design Yourself series to show how i did a glowing neon trail. The video is heavily themed around robots, and if you look in the background you’ll see that it’s actually a circuit board.

The circuit diagram was a random one I built using the rather excellent Fritzing software. If you’re ever looking for high quality SVG illustrations of electrical components then Fritzing is a great resource. I brought the exported SVG diagram into Blender to illustrate it a bit.

If you look closely you can see that the circuit board has a glowing trail. To achieve this effect I followed this tutorial:

At around 6:00 the author tries to find the point where the neon trail position loops but does it visually. At first I was doing the same but then I remembered that in the past I had faced a similar problem when trying to loop the Wave texture. To get an answer for that question I consulted the Blender Stackexchange site

I adapted this a bit and came up with the following solution: To seamlessly loop the neon trail effect first insert keyframe with the Value of the Add node set to 0. Move to a point along the timeline where you want it to loop and add another keyframe to the Value of the Add node and type (0.3333*pi)/$scale (but replace $scale with whatever the Scale of the Wave texture is). My node setup is the same as in the video but here it is as well:

click to embiggen

Now when you play the animation the neon trail effect will loop seamlessly!

Typewriter Text Effect Revisited

For the fifth video in the Design Yourself series I was faced yet again with the task of doing a typewriter text effect. Yay… For each of the videos the participants wrote a poem to go with it. The poems were a really important part of the video so they needed to have a prominent role in the video beyond standard Youtube subtitles. At the time I was producing the video I didn’t yet know whether or not I wanted to use the typewriter text effect but I certainly wanted to explore it as a possibility. One of the first times I tried to achieve this was back in 2019 when I was making the video to promote the Algorave at the British Library.

Since making that video, to add subtitles to a the second Design Yourself video I was using a Natron plugin that would allow synced playback of an audio file via VLC. By using this approach I could seek through the audio and add keyframes to the Text node when the text changed. This mostly worked but sometimes the audio did go out of sync. At some point afterwards I wondered if I could offload the editing of the subtitles – and maybe even subtitles with a typewriter text effect – onto another more specialised program and then import that into Natron later for compositing. Subtitle editing software already exist and are much better suited to editing subtitles than Natron.

I explained this on the Natron forum to one of the developers and some time later the Text node gained the ability to read an SRT subtitle file! An SRT file subtitle file format that is widely used on video files. If you open one up you can see exactly how it works:

The way the SRT reading function works in the new Text node is it basically gets the time stamps of the text in the SRT file and assigns them to keyframes in the Text node.Yay! 🙂

The next stage in the equation was to find a subtitle editor capable of doing a typewriter text effect. There are several open source solutions out there including Subtitle Composer and Gaupol. One of the available software, Aegisub, has quite the feature set. It has a karaoke mode which, as you would expect, can let you edit the timing of words as they appear on screen.

This sounds like the solution to my problem of getting a typerwriter text effect but there’s one big problem. The karaoke text mode only works if the file is exported as an .ssa file format. The SubStation Alpha file format supports lot of formatting options, including typewriter-like text effects. This is good except for Natron only supports .srt file format, and even if it did support .ssa files, I’d still want control over formatting.

To make this work in an SRT file what I needed was for , at user defined poitns, each word to be appended. For example:

1
00:00:00,000 --> 00:00:01,000
Some

2
00:00:01,000 --> 00:00:02,000
Some BODY

3
00:00:02,000 --> 00:00:03,000
Some BODY once

4
00:00:03,000 --> 00:00:04,000
Some BODY once told me

5
00:00:04,000 --> 00:00:05,000
Some BODY once told me the

6
00:00:05,000 --> 00:00:06,000
Some BODY once told me the world

7
00:00:06,000 --> 00:00:07,000
Some BODY once told me the world is

8
00:00:07,000 --> 00:00:08,000
Some BODY once told me the world is gonna

9
00:00:08,000 --> 00:00:09,000
Some BODY once told me the world is gonna roll

10
00:00:09,000 --> 00:00:10,000
Some BODY once told me the world is gonna roll me

It was clear that at this time the Aegisub SRT export can’t do this so in the meant time I made a feature request and reverted back to using the method I described in July 2019 Development Update which makes use of Animation Nodes.

But even then I had a few issues. To synchronise the text with the speech I was expecting that I could use keyframes to control the End value in Trim Text node. However, as explained by Animation Nodes’ developer, the keyframes of custom nodes aren’t visible on the Dope Sheet and so animate anyway. To get around this I used the suggestion by the developer:

I suggest you create a controller object in the 3d view that you can animate normally. Than you can use eg the x location of that object to control the procedual animation.

This worked in the viewer but not when I rendered it. What I instead got was just one frame rendered multiple times. This problem has been reported in Animation Nodes but the solutions suggested there didn’t work. Neither did doing an OpenGL render.

However, I came across this interesting bug repot . It seems like the crashing I was experiencing back in 2019 is a known bug. The solution suggested there was to use a script to render the project instead of the built in render function.

import bpy

scene = bpy.context.scene
render = scene.render
directory = render.filepath

for i in range(scene.frame_start, scene.frame_end):
scene.frame_set(i)
render.filepath = f"{directory}{i:05d}"
bpy.ops.render.render(write_still = True)

render.filepath = directory

The downside of this script is that it doesn’t show the animation being rendered. It also doesn’t allow you to set the start or end point of the render but that is easily accomplished by changing the range in line 7.

After all of that I was able to render the text with the rest of the video! Finished video is below:

I’m getting one step closer to being able to easily create and edit typewriter text using open source software…

Design Yourself: I Never Found Her

In the sixth and final workshop of our Life Rewired inspired Design Yourself project, the Young Creatives worked with artist and writer Erica Scourti.

Erica shared her practice with the group and explored Optical Character Recognition software and speech to text translation processes, to interrogate how identity and human understandings are influenced by these now everyday filters.

Cosima Cobley Carr worked with fellow member and composer Pietro Bardini on a soundscape using a sax line shared by Nayla Chouaib. The video shows us phone portraits of Vangelis Trichias along with Cosima, lip-syncing along with text to speech software. Pietro Bardini created the backing soundscape by taking Nayla Chouaib’s saxaphone recording and taking it through several layers of resonators, reverbs and distortions.

More information here: https://www.barbican.org.uk/read-watch-listen/design-yourself-i-never-found-her