Gifs in Pure Data

Every so often on my travels across the information superhighway I come across a Pure Data user asking if animated gif files can be read in Pure Data. Technically speaking they have always been able to be read in Pure Data, but not always in a way that a user usually wants. Using the [pix_image] object a user can read almost any image file format. On Linux this is dependent on ImageMagick, so whatever it can read can (theorectically) be displayed in Pure Data/GEM. The problem arises because [pix_image] doesn’t display animated gifs as animations, only the first frame.

There are several solutions to this problem. For these examples I’m going to use the following two gifs:

box

frame

Click through each image to get the full-sized original versions.

[pix_multiimage]

If you separate the gif into its individual frames you can use [pix_multiimage] to display each frame in succession.

multiimage
Click to download the PD patch.

Benefits

The benefits of using [pix_multiimage] to simulate an animated gif are that you can display high quality images with an alpha channel at whatever frame rate you choose. Simulating stutter effects or reversing is as easy as using a [counter] or random number generator.

Drawbacks

The problems with this approach are that [pix_multiimage] needs to be told how many frames to cycle through, and not all gif animations have the same amount of frames. [pix_image] and even [pix_data] do not report information about the amount of frames in an animation, so that value cannot be passed to [pix_multiimage]. Assuming that you separate your gifs to their individual frames, an abstraction can be built that can detect how many images there are in a directory and then send that value to [pix_multiimage] but that is a lot of effort to go through!

Convert gif to video

The technique that perhaps most PD users have used is to convert the gif into a video file and use [pix_film] to play it. I used the following script to convert a folder full of gifs into mp4 files, with all transparent pixels converted to green pixels:

With the gif now converted to a video you can use [pix_film] to play a video as you normally would.

gifchroma
Click to download the PD patch.

Benefits

So far I have only tested playing animated gifs in Pure Data using Gmerlin on Ubuntu. Without knowing if the same would work on Windows or Mac OSX, using video files is the safest option for all users.

Drawbacks

Any sort of file conversion will reduce the quality of the output, and this method is no exception. The videos aren’t very sharp, especially at the borders of the green pixels.

Making the green pixels transparent using [pix_chromakey] or [pix_alpha] requires fine-tuning to ensure that other colours aren’t made transparent. This isn’t always 100% reliable and can have a few glitchy artifacts.

Using gifs directly with [pix_film]

Another approach is to use [pix_film]. “Hold on” I hear you say, “[pix_film] can only be used to play films! How dare you suggest that it can be used to play image file formats. Balderdash!”. Well, don’t beleive the hype! As a Linux user, I can only comment on this working on Linux. If anyone can get the following methods to work in any other OS please get in touch and I’ll add it here.

When you play media file formats in Pure Data on Linux you’re actually using external programs and libraries to play them. So, you’ll use ffmpeg/libav to play videos and Imagemagick to display images. There’s also another program you can use, Gmerlin. Install it by executing sudo apt-get install gmerlin. Pure Data/GEM has some weird behaviour whereby the delay amount of a gif needs to be explicitly set to a value 1 or above in order for an animated gif to be played. This can be achieved on a folder full of gifs by executing mogrify -delay 1 *.gif.

And now you can easily open an animated gif in Pure Data the same way you would a video file.

gifvideo
Click to download the PD patch.

Benefits

Gifs, unlike (most) video file formats can have an alpha channel. Another benefit is that you don’t need to deal with converting files. No longer will you have to worry about whether an mp4 is faster or more effecient than an mp4, or what codec to use. Gifs will just be gifs.

Drawbacks

If the original format of your source file is a gif, then perhaps it is more efficient to keep it as a gif. If it was a video file, would it be beneficial to convert it to a gif? Not always. Even if you could achieve a smaller file size or have PD use less processor power by using a gif, the quality of the video output would be reduced due to gifs only allowing 256 colours.

It’s pronounced “gifs”

There are perhaps other benefits and drawbacks to each approach that I haven’t written about or haven’t even thought about. One such example of both is processor usage of each method. I suspect using gifs is actually less efficient, but I don’t have a good method of testing this. Perhaps one of y’all could!

pdroll

Pixel Player

Back in June 2014 I wrote how that in 2013, after visiting The Cyborg Foundation in Barcelona, I became interested in exploring sonification. My experients at that stage culminated in the production of the Pixel Waves Pure Data patch, which allows the sonification of images based on the colour/RGB values of individual ppixels.

I spent the following months building and refining an update to the Pixel Waves software, with a focus on allowing multiple images to be played simultaneously. In a way, I wanted to create a sequencer but for images. After many months I’m happy to formally announce the release of the Pixel Player.

pixelplayer

This software operates in a similar way to Pixel Waves, but with a focus on playing multiple images simultaneiously. Instructions on getting started:

  • Create the GEM window
  • Click on the red button to load an image. Supported file types depend on your operating system, but generally jpg, gif and png file formats are supported
  • Click on the green start button and the pixels will start to be read
  • Drag the orange horizontal slider up to increase the master volume
  • Drag the orange vertical slider up on each pixel player to control its volume
  • Turn the knob to scale the pitch of the audio

The currently displayed/sonified pixel for each channel will be synchronised from the first channel. For this reason it is recommended that all of the input images used are the same dimensions.

This may sound like a lot to do but it becomes easy after a few attempts. To make things easier the loadimage.pd patch has inlets that you can use to control each channel with a midi controller, keyboard, or any other device. To expose the inlets increase the canvas size of the patch by around 10 pixels.

The software includes a video display output, which shows the current pixel colour. This can also be shown on the patch window by clicking the red display button. Flashing lights might not be to everyone’s taste, so this can be turned off. Due to this patch relying on [pix_data], the GEM window needs to be created, even if the pixel display isn’t used.

Enough yapping, what does it actually sound like?! Here’s a small demo, made using a combination of 40×20 images made in Inkscape and images modified using the Combine script by James Allen Munsch (made for Archive Remix. Remember that project?).

Please do give the patch a try and let me know what you think!

An Introduction to Pure Data – 5th August

On August 5th from 13:00-17:00 I’ll be running a Pure Data workshop at Vivid Projects.

puredataworkshop

Pure Data (aka Pd) is an open source a Visual Programming Language, similar to the likes of vvvv and Max/MSP. Pd enables musicians, visual artists, performers, researchers, and developers to create software graphically, without writing lines of code. Pd has seen various implementations including live visuals (VJing), electronic music and even as an embedded library on websites, games or a Raspberry Pi.

This 4-hour workshop looks at using Pd for generative and interactive visuals. Participants will learn how to use Pd to create dynamic motion graphics that react and interact with a variety of sources including audio, live data feeds and hardware controllers.

Logistics

  • The workshop will take place at Vivid Projects, which is located at 16 Minerva Works, 158 Fazeley Street, Birmingham, B5 5RS. Ring the buzzer to be let in.
  • The workshop takes place on Tuesday 5th August and will begin at 13:00 and finish at 17:00.
  • The workshop is limited to 15 places
  • Tickets cost £10 (plus booking fee) and can be purchased from http://www.eventbrite.co.uk/e/pure-data-workshop-tickets-12365061231

What you will need

  • Pure Data Extended. Pure Data is free and open source software that can be downloaded for Max OSX, Windows and Linux from http://puredata.info. The software comes in two versions, Pure Data Vanilla and Pure Data Extended. For this workshop participants should download Pure Data Extended before the workshop begins in order to have more time developing skills and software.
  • A laptop. Although Pd can run on old hardware, newer hardware results in a smoother usage experience.
  • Ideas! After the basics have been covered the workshop will focus on developing any ideas you have.

Thoughts on live coding visuals in Pure Data

I took part in Algorave in Gateshead on 26th April. Apart from being incredibly awesome it was my first time live coding – or rather live patching – visuals in Pure Data from scratch. I emphasise from scratch because nearly all of my performances involve me modifying patches, but never starting with a completely blank canvas. I also occasionally used the HSS3jb as a texture for objects, but never on its own. It’s also great for when crashes occur, which is/was often ;-). Here’s a few sampels of my visuals. Videos by Mariam Rezaei:

I learnt a few things about Pure Data that night, and my general opinion is that it isn’t that great as a live coding visuals tool.

One of first issues is encapsulation of objects. This can be done quite easily but it’s a manual process which would involve cutting all cords and reconstructuring a patch. That is, you would have to cut the selection of objects, paste them into a sub patch and then reattatch it. By way of comparison, Max/MSP has this as a feature already, whereas this isn’t mentioned at all on the bug tracker Feature request is now on the bug tracker. Not being able to auto encapsulate objects makes reuse a bit more difficult and cumbersome, which resulted in some really messy patches from me on the night.

Algorave patches

This also relates to another issue of object insertion. When I was building my patches I would often have to preempt what I would need. I nearly always started with [gemhead]-[translateXYZ]-[rotateXYZ]-[repeat 10]-[rotateXYZ]-[translateXYZ]-[color]-[cube]. Inserting any additional objects required me to cut the cord and therefore the screen output. This would be solved if there were, for example, a method whereby if two objects were selected, the next object was inserted in between them. This is obviously an over-simplified specific use case which would need more thought behind it. Again, no mention of it on the bug tracker. Feature request is now on the bug tracker.

There were other thoughts I had on the night, such as the incosistencies and clumsiness of using the [repeat] object, the lack of a snap-to-grid option for aligning objects, the tiny size of inlets and outlets – even when the ojbects themselves may be huge, which is only exaggerated when using a 13″ 1080p screen, and the lack of a toolbar (yes, I am aware of GUI plugins), but these are the two which I felt would’ve helped me most.

Has much else been written about the use of Pure Data for live coding visuals?

NeonPlastic

I’m happy to announce the release of NeonPlastic, a generative Pure Data artware piece by myself (visuals) and Joe Newlin (audio), inspired by Neoplasticism and all things boxy.

The above video acts only as a preview. To experience this piece in all its hdmegaawesomeness grab the code, open Pure Data, get yourself a cup of tea and press the big red button!

NeonPlastic

NeonPlastic

NeonPlastic

NeonPlastic

Pure Data File Killer

On one of my frequent journeys on the information superhighway I stumbled across Little-Scale’s Mass JPG Killer. This handy little patch allows a user to load any binary file and “glitch” it by overwriting some of the original data with a repeating pattern of user-defined data.

Mass JPG Killer by Little-Scale

The only problem (for me and people like me) is that I don’t have Max/MSP and can’t install it on Linux, meaning I’ve never actually used it!

Little-Scale very kindly provided the internet at large with screenshots of the inner workings of the patch. I was able to to use a whole lot of science and maths to rewrite and reinterpret this patch of mass destruction in Pure Data, which is more easily available.

Pure Data File Killer

Click to download

Click to download

Usage

Usage of the patch is very simple and can yield some quite interesting results!

  • Click open to load a binary file. Pure Data may freeze for a moment if you’re loading in a large file. I don’t recommend loading in a file over 100MB
  • Set the byte offset. This number represents the starting point at which the patch will start “corrupting” the file. If you’re a glitchspert (glitch + expert) you’ll remember that you should avoid modifying the header. To avoid modifying the header set the offset to the 1000s.
  • Set the period value. This can be hard to understand, so here’s an example: If the period is set 1378 then at intervals of 1378 bytes from the offset it will modify the data.
  • Set the data value. This works in conjunction with the period value. Using the previous example, if the data is set to 102 then at intervals of 1378 bytes it will replace the current byte value with 102.
  • Press either random period or random byte data to populate these values with random values.
  • Press glitch it!. Guess what that does.
  • Write the files to save them to the same directory as the source file. The original file will not be overwritten.
  • To start again press the reset button. It will load the original byte data.

This patch is very similar to Little-Scale’s with a couple of exceptions:

  • The offset cannot be set for each instance. This is by design as I felt it was a bit redundant.
  • You no longer need to copy the hex data to a new file in order to view the results
  • It’ll work on any platform that can run a full version Pure Data Extended. This should include the Raspberry Pi version as GEM is not required.

Output

Although it was originally inspired by the JPG Killer you can get some very interesting results if you use other file formats and set the period data to a number less than 20.

Pure Data File Killer - Bliss (sgi)

Pure Data File Killer - Bliss (jpg)

Pure Data File Killer - Bliss (pix)

Pure Data File Killer - Bliss (pix)

Multimedia Programming with Pure Data

A new book by Bryan Chung, Multimedia Programming with Pure Data was recently published by Packt Publishing.

Multimedia Programming with Pure Data

Multimedia Programming with Pure Data

Multimedia Programming with Pure Data

Despite it being a big part of Pure Data Extended, GEM – and making visuals in PD – doesn’t get as much attention as audio processing. Whereas sound-makers have resources such as Loadbang and excellent tutorials from Obiwannabe, visual artists have little access to such a comprehensive resource, which can be a bit off-putting for new users. With that in mind I was more than happy to be a reviewer for this book that focuses almost entirely on GEM and making visuals in PD.

Although it is definitely suited to new users this book does get quite complex in later chapters where it begins to detail camera tracking, OpenCV and particle generators. I even learnt a couple of things!

Most of the tutorials are written to work on all operating systems (Linux, Mac and Windows) though some instructions, such as installing libraries, aren’t always covered. That could be another book in itself!

Get yourself a copy now!

Pure Data Play

Thank you to everyone that attended the Pure Data Play workshop on 2nd November as part of Flip Festival. In the space of two hours the participants went from knowing nothing about Pure Data to manipulating 3D objects on screen, playing videos and webcam streams and controlling their videos using user-defined keyboard shortcuts. Some images of the patches:

Pure Data Play - Alex Jolliffe

Patch by Alex Jolliffe


Pure Data Play - Eliza Marcu

Patch by Eliza Marcu


Pure Data Play - Jamie Boulton

Patch by Jamie Boulton

To those of us who know more about programming and using Pure Data these patches may seem simple, but hopefully from this tutorial the participants have gained an insight into what is possible using Pure Data.

What I like most about Pure Data is that it is very extensible. It can accept data from a wide range of sources – including Arduino boards, game controllers (including Wii remotes and Kinect controllers), microphones, lists of data and even raw binary data – manipulate it and give audio or visual feedback. Best of all it does it in a way that is very logical. Some people may prefer to write lines of data, but with Pure Data (and other dataflow languages) you can visually see how data flows and is manipulated.

If you’re interested in attending or booking me for a Pure Data tutorial get in touch!

Pure Data Play, Friday 2nd November

On Friday 2nd November from 12pm I’ll be running a Pure Data workshop as part of Flip Festival at Lighthouse Media Centre in Wolverhampton.

pd

Pure Data is a dataflow programming language that is utilised for a wide variety of purposes including making generative electronic music, creating glitch visuals, interactive live performance and VJing (Video Jockeying). I’ll guide participants through the basics of Pure Data – including general usage, playing and manipulating videos and creating generative visuals that respond to different inputs. No experience of Pure Data is necessary, only a laptop and a willingness to experiment and learn!

The event is free to Flip Festival ticket holders and £6 for everyone else. Tickets can be purchased online. Here’s the event details on Facebook. Don’t forget to bring your laptop!

To see what can be created with Pure Data take a look at what I’ve made in the past.

Create jpgs in Pure Data

For Some of My Favourite Songs I utilised Pure Data Extended (I’m using a beta version) to read the audio files and then save them as images. Pure Data is usually used for the production of music and/or generative live visuals, so to using it to produce jpg images from almost nothing, or random data input is quite new to me!

In search of a jpg header

The most important part of this process is knowing how to construct and apply a jpg header to data. Wikipedia informed me that all jpg images begin with FF D8. I thought that all I would need to do is use a hex editor, such as Ghex or Bless Hex Editor, to add those byte values to a file.

Unfortunately this is not the case at all. There’s so much more in a jpg header, such as Huffman Tables, Quantization Tables, bytes to define the width and height of an image, and much more that I still don’t quite understand.

I attempted to grab data from the beginning of a random jpg file, but this included lots of extraneous data such as camera make, program(s) used to modify the photo, gps data and creation date. This data amounted to several kilobytes, which is far too much data for a header. What I needed was a “vanilla” or plain header that I could apply to any file.

mesmeon showed me the HEADer REMIX project by Ted Davis. The header values on the left of the screen are used for glitching every image, be it the default image or one taken by a user.

I saved the default image, manually extracted the header image, ran it through exiftool and then ended up with a header for a 640×480 image that is only 588 bytes!

Enter Pure Data

Now that I had a vanilla header I had to devise a way to use it in Pure Data. The [binfile] object allows the reading and writing of binary data. Adding data to [binfile] is a case of sending a message containing numbers to the object.

[binfile] reads and outputs data as decimal values i.e. numbers from 0 to 255. I needed to find a way to add the decimal values of the vanilla header to a message box. Martin Meredith helped me with this whilst we were tackling bugs at the Ubuntu Global Jam. Using hexdump I was able to output all of the hex values to decimal values.

hexdump -v -e '1/1 "%02u "' filename.here > decimalvalues.txt
255 216 255 219 00 132 00 03 02 02 03 02 02 03 03 03 03 04 03 03 04 05 08 05 05 04 04 05 10 07 07 06 08 12 10 12 12 11 10 11 11 13 14 18 16 13 14 17 14 11 11 16 22 16 17 19 20 21 21 21 12 15 23 24 22 20 24 18 20 21 20 01 03 04 04 05 04 05 09 05 05 09 20 13 11 13 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 255 192 00 17 08 01 224 02 128 03 01 17 00 02 17 01 03 17 01 255 196 01 162 00 00 01 05 01 01 01 01 01 01 00 00 00 00 00 00 00 00 01 02 03 04 05 06 07 08 09 10 11 16 00 02 01 03 03 02 04 03 05 05 04 04 00 00 01 125 01 02 03 00 04 17 05 18 33 49 65 06 19 81 97 07 34 113 20 50 129 145 161 08 35 66 177 193 21 82 209 240 36 51 98 114 130 09 10 22 23 24 25 26 37 38 39 40 41 42 52 53 54 55 56 57 58 67 68 69 70 71 72 73 74 83 84 85 86 87 88 89 90 99 100 101 102 103 104 105 106 115 116 117 118 119 120 121 122 131 132 133 134 135 136 137 138 146 147 148 149 150 151 152 153 154 162 163 164 165 166 167 168 169 170 178 179 180 181 182 183 184 185 186 194 195 196 197 198 199 200 201 202 210 211 212 213 214 215 216 217 218 225 226 227 228 229 230 231 232 233 234 241 242 243 244 245 246 247 248 249 250 01 00 03 01 01 01 01 01 01 01 01 01 00 00 00 00 00 00 01 02 03 04 05 06 07 08 09 10 11 17 00 02 01 02 04 04 03 04 07 05 04 04 00 01 02 119 00 01 02 03 17 04 05 33 49 06 18 65 81 07 97 113 19 34 50 129 08 20 66 145 161 177 193 09 35 51 82 240 21 98 114 209 10 22 36 52 225 37 241 23 24 25 26 38 39 40 41 42 53 54 55 56 57 58 67 68 69 70 71 72 73 74 83 84 85 86 87 88 89 90 99 100 101 102 103 104 105 106 115 116 117 118 119 120 121 122 130 131 132 133 134 135 136 137 138 146 147 148 149 150 151 152 153 154 162 163 164 165 166 167 168 169 170 178 179 180 181 182 183 184 185 186 194 195 196 197 198 199 200 201 202 210 211 212 213 214 215 216 217 218 226 227 228 229 230 231 232 233 234 242 243 244 245 246 247 248 249 250 255 218 00 12 03 01 00 02 17 03 17 00 63

(The output is sent to a text file for ease of copy/pasting)

With this output I copy/pasted the values into a message box, and whenever I needed to add a jpg header to a file I clicked on the message box! To then write the file I sent the message [write filename.jpg( to the [binfile] object.

A jpg header in Pure Data

Using this data alone you may notice that the jpg image doesn’t open in certain image viewers or is blank/black. That is because all that was added is the header. Image data is also needed! For this I added a few [metro]s to generate random numbers between 0-255. The output image then looks a little bit more colourful now.

jpg created by Pure Data

For some websites and image viewers the End Of Image bytes (FF D9/255 217) need to be added in order for it to be viewed properly. To start this process again send [clear( to [binfile]. This clears all binary data. Below is all of this theory put into one patch.

The finished Pure Data patch

Pure Data [binfile]

Generate jpg images – click to download

To use it, first click on the button to start the jpg file, then click the toggle button to add lots of random data. This may take a minute or so. Once done turn off the toggle, click on the button to end the file and then write the jpg image.

Further options

If you know the structure of a certain file type, in theory, it is possible to construct one in a similar way to this. I’ve already used this method to construct a bmp, but they produce far less interesting results. png files seem to be more fragile and, as such, I haven’t managed to create one using this method.

If you use a second [binfile] object you can load the bytes from another file and use them, in conjunction with random data, to produce glitchy – but slightly recognisable – images!

Is it also possible to reduce the size of the jpg header even further?

Create jpgs in SuperCollider

Holger Ballweg (uiae) has recreated this progress in SuperCollider. Check it out!