MoMA Display Screens

“MoMA Display Screens” is a real-time video installation that is on permanent display in the lobby of the new MoMA in NYC. It is a 9-channel video system, composed of nine 30” HD LCD monitors and a network of 10 computers.

The content is generated live. Images and actions appear across multiple screens. If you watch the screens indefinitely, you will never see exactly the same thing twice.

The system algorithmically selects from a catalog of still images, then selects from a repertoire of action sequences to apply to these images. Actions occur on anywhere from one to nine screens. Multiple actions occur simultaneously across the image field.

The museum regularly provides the system with new still images of temporary shows and the permanent collection, as well as new text information generated daily, so the content displayed is always new.

“MoMA Display Screens” is a project of MoMA, design firm Imaginary Forces, sl.inc and Kurt Ralske. I designed the system architecture, co-designed the action sequences, and was solely responsible for implementing the image processing code.

 

Design of the system

            The system is a network of 10 computers. There are nine “player” machines, one for each screen.  One “master controller” machine controls and synchronizes the actions of the player machines. There is a pool of “show templates”, which are XML definitions of a sequence of timed actions to be performed to an unspecified set of images (eg., fast blurring, slow scaling, cross-fading, etc.). There is also a pool of “show definitions”, which are XML definitions for a set of specific images, as well as which show template (action sequence) should be used for that image set. In this way, any set of images can flexibly be associated with any sequence of actions.

            The master controller combines the show definitions with their associated show templates and positions the image sets to fill the 9 screens. Selection and positioning of groups of images are done algorithmically, according to a set of rules expressed as weighted probabilities. The weights are determined by the size and duration of each show and each show’s assigned relative importance, as well as other external factors such as the time of day.

            The following technologies were used in the project:

C -       for graphics processing code, message parsing, action timing
Java - for XML parsing and show scheduler logic
XML - for storing show templates and show definitions
Max - for image display and image loading, and as a wrapper for C code
OSC - network protocol (a UDP protocol)

 

Challenges in realizing the project, and how they were met

Problem: How to load huge amounts of high-resolution video, without hard drive latency?

Solution: The original specification called for playback of rendered video from the hard drive. But it was determined that this was not reliable at the high resolutions required by the project.  An alternate solution was devised:

1)     Instead of using video as source material, still images would be used. This would greatly reduce the amount of data loaded from disk.

2)     Visual motion would be created by moving, transforming, scaling, and blending layers of still images. These actions would be created live, in real-time.

 

Problem: How to constantly load large still images from disk, while avoiding interfering with processor-intensive real-time image creation?

Solution: It was determined that performance was improved by loading all necessary images into RAM at a few intervals during the day. In this way, it is no longer necessary to load images and process images at the same time.

 

Problem: Given that every player uses a different set of images, and that certain images are more likely to occur on some players than others, how can the master controller machine catalog which images are on which machine?

Solution: This problem was side-stepped by deciding that any image used by any machine should be loaded by all the machines. In this way, the master controller can use a single catalog to retrieve images from all the machines. The name used to retrieve the image is the same regardless of which machine is addressed. 

 

Problem: While performing complex processing and layering of images, how to keep performance high? How to prevent poor frame rate or dropped frames?

Solution: Following suggestions from several books on code optimization written in the 1970s, these guidelines were followed in the image processing code (as much as possible):

            Using integers instead of floating-point

Fixed-point math

No divisions

Bit-shifting for multiplication and division

Keeping the inner loop free from math

The goal (which was achieved) was to reduce any graphic process (eg, scaling, blurring, blending, etc.) into an operation involving pointer arithmetic only.