Eternalism & the art of slitscanning

Posted by Amnon on January 16, 2011 · 22 Comments 

Eternalism is a philosophical view on reality that proposes time is just another dimension. As opposed to the view of the world as a three-dimensional space modulated by the passage of time. An interesting facet of eternalism is that it implies future events are “already there” and as such that there is no objective flow of time. One can argue about the truth of such a theory in the real world. With regard to the world of film, it is most certainly true. When the viewer starts the film, the ending is already there. So the perceived spatial dimension and the perceived temporal dimension, are truly equal. This creates the possibility to view everything in a non-conventional way.

The manipulation of time
Wasn’t this blog about Processing? Yes it is! So let me get right to point. Time bending experiments in Processing, the possibilities are endless! Visually that is. I’ve been working on a series of sketches that mess with the flow of time in some way, shape or form. Ranging from manipulations at the frame level, all the way down to the very pixels that images are made of. I have no idea what you would normally call these effects. But for myself I used names like TimeMatrix-based VideoSequencer and pixelTimeDisplacement (preliminary screenshot below). More than just cool names, they’re actually a fairly decent representation of the functionality. Perhaps later I will release, write or show material related to these experiments. The focus of this blog post however will be on the technique I used to make the video above.

Slit-scanning
The term slit-scanning has it’s roots in the history of the technique. Back in the days, the process would supposedly be as follows. A moveable slide, into which a slit had been cut, was inserted between the camera and the subject to be photographed. Making one single slit-scan photograph would thus require a whole series of shots. You can imagine this was extremely time-consuming. Over the years, technological advances have not only made it cheaper, easier and faster to create slit-scan photography, but also allowed for creation of and experimentation with slit-scan video art. A great catalogue of the latter is maintained by Golan Levin. So check out this page to see some inspiring examples. You can also find a simple slitscanning sketch for Processing there, which works very similar to the old-school techniques I just described.

Existing code
We have computers now to do all the heavy lifting (yay for the digital age 8)). And there are quite a few code examples out there to get started with slit-scanning. Although a good starting point, none of those were unfortunately completely suited for what I had in mind. Which was full HD slitscanning of existing image sequences. The two biggest problems for me with existing sketches: they were fairly slow to process frames or incapable of processing large files altogether due to memory restrictions. So these were the two points I focused on when developing different prototype sketches. What I ended up with ain’t perfect, but it’s good enough for the job at hand. Also, the learning process itself is still my most important goal. Of course solid software and pretty pictures are a big bonus! The final sketch works in batches and uses the maximum amount of memory it can use for a single batch. So it works regardless of the size and number of images. A small video can be processed in a matter of seconds in a single batch. If you’re processing a longer sequence of full HD images, it can take around two hours, depending on your computer and memory limit. However in my opinion this is pretty reasonable. Plus the added advantage of batches, is that after the first batch (10 minutes even with full HD) you’ll already be able to see the first series of resulting images. And decide if you wanna see the rest.

SlitsP5
In the spirit of open-source I like to share the code I wrote. For lack of a better name, let’s refer to it as SlitsP5. You can download it from here. Of course I release my code under the usual disclaimer. That it was created as a personal experiment by yours truly. So there are no guarantees it actually works, haha. For your convenience I placed almost all of the code in a class, so it becomes really simple to use it. Aside from the aforementioned class, the code example below is the only thing you need to run it (this is all included in the download, no need to copy-paste). As you can see there aren’t many settings, just state the directory and the direction of the slit-scanning (X or Y) and that’s basically it. Some settings hidden in the class itself are a boolean for saving (true by default) and one for additional console output (false by default). I put them there, cause they’re unlikely to be used anyway. I’ve experimented with video and image sequences for both input and output. But in the end I found video unreliable (fps, codecs and whatnot), so I switched fully to image sequences for both input and output. For me there are a number of advantages to this like accuracy, flexibility and quality. The downside being the need to separately convert between video and image sequence. Of course it’s possible to rework this sketch to include direct video input and output.

SlitsP5 mySlitscan;

String directory = "inputDirectory";
char direction = 'X';
int memBarrier = 320000000;

void setup() {
 size(500,500);
 mySlitscan = new SlitsP5();
}

void draw() {
 mySlitscan.run();
}

Setting the memBarrier
With regard to settings, the only really important thing is to correctly set the memBarrier. You only need to do this once. The memBarier is the maximum amount of pixels your computer can hold in the memory at one time (given the Processing/Java/WinXP memory limitations). So this is different per computer. For me it was about 360 million, so I set the memBarrier at a secure 320 million. A little less won’t have a huge impact, but if you set it too high, you’ll get an out-of-memory error right at the start. I’m pretty sure there is a neater way to manage memory, but since you only need to do this once, it’s not a whole lot of work. So how do you find your memBarrier? Well, for your convenience another sketch – called memBarrierTest – is included in the download. Run this program until it crashes. Yes, crashes. Then scroll up a little in the console and see the last line that says something like “Number of pixels in memory: 363.250.000”. That’s the number you can use for the SlitsP5 sketch. Just to be safe you can even set it a little lower.

How to use SlitsP5
Using SlitsP5 in four steps (assuming a correct memBarrier is set):

1. Place the input image sequence in a subdirectory in your sketch directory.
2. State the name of the directory in the sketch (images will be read automatically).
3. State the direction you want to slit-scan (X or Y).
4. Run. You will see information and a progress indicator in the console as well as the main window. Output images are saved in a subdirectory of the same name as the input directory with the addition of either an X or an Y, depending on your slit-scan direction.

Tips, tricks & troubleshooting
A collection of useful suggestions to help you on your way:

• SlitsP5 automatically reads JPG, PNG and TGA images from the input directory. If you want to use other image formats, it’s fairly easy to add them to the class (as long as they can be read by Processing in the first place). You can add other filetypes in the loadFilenames() function.
• Remember that the number of frames will be your new X or Y dimension. For example if you have 100 input images of 200 x 200 pixels and you slit-scan over the X. You will end up with… 200 images of 100 x 200 pixels. That is slit-scanning. A spatial dimension switches places with a temporal dimension. So if you wanna end up with full HD width (1920 pixels) you will need to input exactly 1920 frames.
• Footage with big cuts and/or compilations doesn’t work well. As you can imagine, sudden changes stand out in slit-scanned material. Slow moving objects on the other hand create much smoother output.
• If you get an outOfMemory error right at the start, you need to correctly set the memBarrier!

Inspiration
For anyone interested in slit-scanning, I can recommend the 1988 experimental short film The Fourth Dimension by Zbigniew Rybczyński, a filmmaker and innovator in the technical field. Aesthetically the video I made was also influenced by the work of Pipilotti Rist, an amazingly talented video artist. I was – unexpectedly – introduced to her wonderful art on my visit to the Fundació Joan Miró in Barcelona. Last but not least an obvious source of inspiration were the works of previous Processing tinkerers such as Don Whitaker, Lukasz Karluk and Matt Ditton. Don Whitaker also happens to be the man behind Mother Nature Videos. This is one of the few online collections of free quality HD stock footage, which is perfect material for slit-scan experimentation.

Looking back, looking forward
It’s becoming kind of a blog post tradition to reflect upon the whole creative & educational process. With each project, I just feel that my grasp of programming in general and Processing in particular is growing. Which is of course great. I do notice that despite my best efforts, my to-do-list is even growing faster! The more you know, the more you realise you don’t know. There are numerous areas of interest that I’d like to investigate. I just try to go from one to the next intuitively, so we’ll see what happens… I hope this post, video and/or source code is useful to someone out there. And please feel free to talk back in the comments below.

You can download SlitsP5 from here.

Advertisement

Filed under Animation, Experiments · Tagged with 2d, animation, chronotopic anamorphosis, color blending, computational art, processing.org, slit-scanning, slitscan, source code, time bending, time dilation