Blinkenlights Stereoscope

Introduction

From July to October 2008 I was involved in one of the coolest digital media projects in the world: Blinkenlights Stereoscope. Now that I've recovered from the late nights and gotten back to my normal life after several months of craziness, I thought I'd write a full account of my experience since a lot of people have asked me about it. This is a fairly non-technical version since there will be more technical information about the hardware and software made available on the Blinkenlights site before too long.

First a bit of introduction for those of you who might not know what I'm talking about. In 2001 and 2002 some very cool computer hackers from the Chaos Computer Club in Germany took over a building in Berlin, and a bit later one in Paris, and installed lamps behind all the windows and made a giant low-res video screen out of the entire building. The first one only had 144 windows/pixels and could simply turn the lamps on and off, the second had over 500 pixels that could operate in eight brightness levels. Using a mix of custom games, animations and other programs these buildings displayed low-res images and animations which dazzled the public. People could interact with the displays through their cell phones to play games, or by contributing their own movies to play on the buildings. The Paris installation was part of the very first Nuit Blanche festival in 2002. The Nuit Blanche festival concept has since been copied around the world. Toronto had their third Nuit Blanche festival in 2008.

The Blinkenlights crew were careful to document many aspects of their original installations and released some documentary videos online which you can download or find on Youtube. I came across these videos a few years ago and wanted to do something similar in Toronto. This was before we started to have a Nuit Blanche here, and the idea sort got put in the back of my mind until this past year when I revived it and started designing a different sort of large media exhibit for Nuit Blanche inspired by, but not a copy of the Blinkenlights project. However, my skills are in technology and not money or politics so my concepts never got off the ground. Once again the idea got put in the back of my brain.

But in July of 2008 as luck would have it, someone in my computer club posted a link to an article published on the CBC.ca news site claiming that the Blinkenlights crew had been invited to do an installation at Nuit Blanche in Toronto in 2008. When I read this I was in Paris, oddly enough, nearing the end of my summer vacation. I sent email to the contact mail on the Blinkenlights website offering my help. The site hadn't been updated in years so I wasn't sure if anyone would get my message, but sure enough I received a response almost right away. As it turned out the two main organizers of the project: Tim and Fiedel were going to be visiting Toronto two days later to meet with the Nuit Blanche planners. I was scheduled to return to Toronto the very next day so we agreed to meet when they got into town. We had an excellent meeting over dinner and they mentioned some new ideas they wanted to implement but didn't have time to do themselves. We agreed that I would I assemble a small coding group so I recruited the help of my long-time friends Dan Fraser and Jonathan Fuerth. We (the Canadian Team) met with Tim and Fiedel several more times over the following few days to discuss the project in more detail.

Surpassing both Berlin and Paris, the Stereoscope installation had nearly twice as many pixels installed in two buildings. The distinct Toronto City Hall with its two curved towers was the chosen venue. These two buildings are really divided into four sections because there is a mechanical floor in the middle of each building which doesn't contain any windows. There are a total of 960 windows in both buildings which would each be lit by a 150W halogen flood lamp. (that's 144,000 watts!) But unlike previous installations where either the building was empty or had hallways adjacent to the windows, most of the windows in City Hall are part of active office cubicles where people work each day. Clearly this time around there would be a lot of new challenges.

Stereoscope Official Documentary

bmix and bvoip

The majority of work that Jonathan, Dan and I were involved with included designing and writing two very important pieces of software that were desired for the Toronto installation. The first one, bmix was a special sort of video mixer program that was used to blend a lot of different video sources together and split up the mixed image to be sent to a number of different places. This was necessary because there were two buildings each divided into two "screens", the upper and lower portions separated by the mechanical floors. Additionally, because the buildings are curved they form part of what can be considered a large imaginary cylinder. Tim's concept was to render video for a complete virtual cylinder and treat the actual windows as a view into this larger buffer. We called this large virtual cylider the "virtual matrix" and it was defined to be 96 x 32 pixels, which are nice computery numbers and seemed to encapsulate the existing windows perfectly. This meant that if images were moved across the virtual matrix they would appear to leave one building and reappear on the other as if the image had naturally transited through the blank space. Our bmix program would be the master mixer/router that would accept many different video sources, position and scale them onto the virtual matrix, allow them to blend together observing transparency rules (think Photoshop layers) and finally grab various regions from the virtual matrix and send them to various places.

We also eventually wrote a stats client to help better visualize what was happening on the system by showing previews of each mixer input, and resulting outputs including all kinds of useful data. A picture of what it looks like is shown below.

The second program known as bvoip was designed to be a replacement for the original ISDN telephone interface which allowed callers to play games, and team members to control and debug the setup from their phones. It was clear that VOIP was the new perferred way of getting telephone services for the project so we built the bvoip application as an adapter to fit between Asterisk, a popular open-source PBX system and the original video and game server. (known as blccc)

Jonathan and Dan and I met once per week during July and August (and more often closer to opening night) for evening coding sessions, and did some work independantly in our own time. Our coding sessions were all done in the Extreme Programming way with one computer and three programmers. We took turns driving and had the benefit of the 70" LCD monitor in my company's board room so that we could all see what was going on. We often used extra computers for providing data streams or debugging, but the main development almost always happened on only one computer at a time.

All the code was written in Java which was initially a shock to the German team who are mostly C programmers, but it soon became clear that Java not only helped us write better code faster, it also performed much better than most of the original C programs which we were interfacing with.

New Games, Processing, etc.

After getting bmix and bvoip working we started getting interested in making new games, since the only usable ones from the original Blinkenlights codebase were Pong and Tetris. Since we had a lot more screen real-estate we figured it would be fun to have more games to choose from. Jonathan wrote a Java game API that interfaced with bvoip using the original telephone protocol. He and Thomas, one of his coworkers, worked on games and we ended up adding four new games to the system: snake, space invaders, breakout and mole (whack-a-mole) all of which became as popular as the originals.

We also experimented with the Processing programming language and ended up with yet another Toronto coder involved. Robin Senior had offered his help to the German team and ended up writing some code to export Blinkenlights movies from Processing. Dan had written a module to generate live Blinkenlights network streams from Processing so now we had a pretty useful way of making movies and live data with the very powerful and easy Processing programming language. Robin made a cool amoeba animation using Processing, which quickly became my favourite. I wrote a test program for testing lamps, and a bouncing ball which quickly became annoying. Other people had some success with Processing as well, making cool videos and utility programs.

Once we finished all the new Java code we had pretty much learned about every aspect of the movie and network formats used in the original Blinkenlights installations. With reusable code in mind we continued working on other useful utilities and add-ons. Dan wrote a very useful program for taking a bunch of PNGs and turning them into a movie file. We put it up for download and lots of people in the community used it to generate their own movies which they contributed to the project.

There were other new software developments as well which we weren't involved in. A suite of Mac OS X tools using Quartz Composer included a 3D simulator and some conversion tools for allowing Mac users to make their own movies. This was written by the German coders Domenic and Martin (the Coding Monkeys). Part of this project also resulted in an iPhone visualization program which showed a real-time view of the buildings when the event was in progress, and now shows re-runs of the live data. The app is free and can be downloaded from the app store.

The Setup

The Stereoscope setup was the largest in Blinkenlights history and perhaps even the largest of this kind in the world. In the past all of the lamps were connected to the control systems with cables. But the sheer size of this installation: 36 floors with 22 or 30 lamps per floor, it was clear that running cable to every lamp for control would be very expensive and cumbersome. Instead the team decided to use a wireless dimming system built on top of the existing OpenBeacon technology developed several years ago for the Chaos Computer Club conference in Berlin. This ultra low-cost 2.4GHz wireless technology had proven itself in the past so it was used as the basis of a wireless lamp dimmer module. Over 1000 units were manufactured and assembled on an amazing schedule to be ready for Nuit Blanche. The dimmers and transmitting equipment was developed by the OpenBeacon developers Milosch and Brita, and firmware was made by both Milosch and Daniel. Each lamp was a standard 150W flood lamp with a 78mm T3 bulb. The entire lamp assembly was mounted on a small pole which was clamped to the window sill. A large vinyl sheet was installed over each window with small binder clips and elastic bands. Each sheet included an extra clip so that workers could open their covers during the daytime. It took about an hour to close all the covers, raise the blinds, and move the occasional plant from the windows each night during the week-long run of the project.

Getting the data to each dimmer required a small transmitter unit on each floor. This was existing OpenBeacon technology that was loaded with custom firmware. This connected via Ethernet to the network room on each floor where a small power supply powered it (sort of like power over Ethernet) and its Ethernet connection attached to a cable running back to a network switch. Each of the four "screens" (top and bottom blocks of floors in each building) contained a network switch in the lowest floor which was dedicated to Blinkenlights only and had no other traffic on it. The network guy Mark from City Hall was most generous in helping us late at night with our networking needs. Each of these four switches was connected by fiber back to the headquarters for the project on 4 west. We had an entire floor (with lamps) which we used for storage, offices, cooking and our master control operations.

The more I learned about the wireless technology the more I thought it wasn't going to work. And initially my suspicions were correct. So many of these units on the air at once sharing frequencies, noise, and low power (1mW) made me doubt if it was possible to expect 960 wireless dimmers to all work at once. Initially we had a lot of problems with interference between transmitters, buggy firmware and weird misbehaving dimmers. After all, the full setup had never been tested before being installed in City Hall. But after experimenting and learning the problems, Daniel and Milosch cranked for several days on the firmware and came out with a fully working system! As Milosch writes: "The night we got the dimmers working was actually a cool and incredibly effective transcontinental and trans-timezone 72 hours coding session between Daniel and me which I enjoyed very much - before that nothing worked - afterwards everything." That is definitely the sign of true and dedicated hackers!

With 960 dimmers installed in a somewhat random order you'd think it would be impossible to know how to address them all properly. And you'd be right! Luckily there was a plan for this which involved each dimmer having a unique ID number programmed into its ROM, but also the same number printed onto a barcode on the outside of the case. By walking along the window sill with a laptop balanced on one hand and a barcode scanner in the other it took only a minute or two to scan an entire floor. Once this information was recorded each lamp could be told which pixel to be. This data was loaded over the air using special utility programs. Although it wasn't possible to monitor each and every dimmer in real-time we did have a lovely program that Daniel wrote to ping the transmitter on each floor. Most massive failures of an entire floor or block of floors were due to network problems. Individual dimmers behaving badly were easy to see just by looking outside.

Master Control and Video Playback

Like any good show there needs to be a main command and control centre from which everything is run. And Stereoscope was no different. With a couple of large LCD monitors and a few chairs and tables we assembled master control. One screen usually ran the bmix stats client, and the other screen ran Domenic's simulator so that we could see what the actual buildings were looking like. Everything was run from a single Linux computer including most of the movie playback, bmix, bvoip and the proxy tools sending streams outside for people to watch on their PCs or iPhones, and all the diagnostic and setup tools. The telephone server running Asterisk was run from a small 600MHz Mini-ITX computer running FreeBSD 6.3. All the internet for the project was provided by the City Hall people over an Ethernet link. The transmitters for the dimmers had a private LAN only accessible from the Linux computer via a second network card.

Often people would send movies from their laptops around the place by sending videos to various inputs on bmix. Bmix treats each input as a layer which goes on top of others layers. So there were always movies running in the background. When someone would send to an input it would take over the background movie. When they would stop sending the input would time out after a few seconds and start showing the background movie again. Games worked this way too, except that they were set up to be semi-transparent so that the background movie would show through. Since most games have black backgrounds this made things look more interesting.

One very special part of the main Nuit Blanche presentation was the live show presented by Falk. He is an amazing VJ who prepared an entire live show with music to be presented on the night of Nuit Blanche. For the presentation he set up his laptop in Nathan Philips Square and Ethernet was run back to the main headquarters. He generated images in real-time and his program was configured to send Blinkenlights streams over the network to be displayed live. He made great use of the buildings and everyone loved the show.

The People

There is so much more that I could included here, but my short but hopefully informative description wouldn't be complete without a mention of the talented people that not only made this amazing project happen, but also allowed my friends and I the opportunity to work with them. We developed some great friendships which I know will last for a lifetime. And I know that we all learned so much from each other.

Tim and Fiedel are the main organizers of the project and the ones who allowed the Canadian Team to be involved. Tim is an incredible spokesman, planner, project manager, technical visionary and someone who just knows how to bring people together in a way that can result in great things. Fiedel is an amazing technician who just understands all there is to know about electricity and wiring, not to mention having a great time at every moment. The team also included many other very talented people including Daniel, who is one of the most talented electronics and firmware guys around, and Milosch and Brita who designed the dimmers, Domenic and Martin who wrote all the OS X and iPhone apps and Papillon who did just about everything and never seemed to run out of energy.

Conclusion

Although I walked around like a zombie for a few weeks trying to get stuff done at work and also stay up all night doing Stereoscope, I didn't seem to mind one bit. More coffee and pizza and beer was consumed during those few weeks than I care to admit. But when I was standing outside on the night of Nuit Blanche watching Falk's VJ show, all of a sudden I got that elevated high you get when you know something is totally and completely right. That moment made it all worth it and I know that if I had the chance, I would gladly do it all again.

More Info

If you want to learn more about Blinkenlights or the technologies and people behind it, please check out the following links:

• Stereoscope - The main Stereoscope project page
• Scotia Bank Nuit Blanche - The Toronto Nuit Blanche main site, sponsored by Scotia Bank
• OpenBeacon Blinkenlights - The dimmer and wireless communications hardware for controlling the lamps