The Project
The National Museum of American History opened their new exhibit, The American Revolution: A World War, in June 2018. As part of this new exhibit I was tasked with creating two new interactive, touch-screen, kiosks.
This project was first mentioned to me almost two years before I began work on it by the Curator, David Allison. As part of this new exhibit, the 1786 paintings The Siege of Yorktown and The Surrender of Yorktown, by Louis-Nicolas van Blarenberghe, on loan from Nicholas Taubman, would be displayed in the gallery. These paintings depict the Revolutionary War siege and surrender of the British forces at Yorktown in fantastic detail. The team was interested in using interactive kiosks to allow visitors to explore the details much more closely than would be possible by pressing their faces to the glass. We would build one kiosk for each painting. Another goal for these kiosks was to provide additional context from the many eyewitness accounts from the time.
The Challenges
Performance
One of the biggest concerns for this project was performance. User navigation of the paintings had to work perfectly smoothly even with the massive image files required to show the full detail of the paintings. The full resolution image file for each painting was roughly 80 times the size of your standard high-definition computer or TV display. But, it had to move smoothly as the user dragged the image around and zoomed in and out.
User "Enthusiasm" and Museum Durability
Although the programming languages and tools are the same, developing kiosks for museums has a few key differences from developing a standard website.
On the positive side, I only have to build for one specific set of hardware. Instead of having to make sure it runs smoothly on everything from computers to phones all with different capabilities, I can target everything to performing as well as possible on one set of equipment.
However, websites running on museum kiosks are used in a very different way from your typical website online. Users only stay on most websites for a few minutes at a time. This gives many opportunities for things to refresh and potential problems to get cleared out. Our kiosks usually boot and load content at 9:00 each morning and shut down at 5:30 each night. With no page reloads in between. This means no chance to clear out errors. Even things like a tiny memory leak, which would get cleared out on a reload, would become a problem by the end of the day. So, there is even less room for these types of errors when programming museum kiosks.
We also have very, let's just call it enthusiastic, visitors. Most websites only need to respond to one click or touch point at a time. Maybe two if they have a pinch to zoom function. It's not unusual to see groups of as many kids as can fit around the kiosk all trying to interact with it at once. This can lead to the touch screen registering dozens of touch points. If your programming is trying to respond to all of these touch points at once, it can cause all sorts of unpredictable problems. This makes it incredibly important to program things in such a way that it cannot be trying to execute multiple commands at once.
The Process
From the beginning, this was a close collaboration between myself, the curator David Allison, Howard Morrison, who provided the points and quotes for the contextual information, Exhibit Designer Stevan Fisher, and Project Manager Tanya Garner.
Early on, I decided there were too many context points to try to just program them all individually, also known as hard-coding. Instead, I decided it would need to be programmed as a template with the actual data being loaded from a data file and plugged into the template. This had the added benefit of allowing me to use the same programming for the two kiosks and have the painting selected by just a few variables. I decided to use the React framework since it would allow the easiest programming and most flexibility for an application of this type.
The biggest challenge of this project was getting the painting images to animate smoothly. The image files for each painting are massive. Trying to smoothly move the full image around the screen would bring even a powerful computer to its knees. There are two obvious solutions. We could have used a lower quality image. But that would defeat the purpose of having the kiosks to show the full detail in the paintings. We also could have just used a single zoomed in image for each of the context points, and have the kiosk jump between them. But, from the beginning we wanted the user to be able to explore the painting on their own. So, what were our options?
I ended up using a two part solution. For smooth animation and when the image was zoomed out, I used a lower resolution image file. This one is only about 1/81st the size of the full image. For when the image is zoomed in, I split the full image into a grid of 81 smaller images at 1920x1080 resolution that sits over the lower resolution image. When the image is moving or zoomed out, the high resolution grid is hidden. When it stops moving, I programmed a function to use the location and zoom ratio to determine which of the 81 smaller images are currently visible in the frame. It will then load only the visible images and fade those in as soon as they are done loading. This allows the painting to move very smoothly and the full quality image to be displayed very quickly after the animation has ended.
As this project progressed, it was decided that these kiosks should be available on the exhibit website as well. This required a few slight modifications to provide a painting selection page and to provide navigation alternatives on non-touch screen computers. Unfortunately, some of the animations would not run smoothly on mobile devices and were therefore disabled in favor of jumping between points on the painting.
Finished Product
The final result was better than I had ever hoped for. The two kiosks allow the visitor to see a much greater level of detail in the paintings and view a great deal more contextual information than would be possible with the paintings alone. Visitor use of the kiosks has been heavy but they have performed stably so far.
The online version of the kiosks can be viewed on the National Museum of American History website here. The Siege of Yorktown and Surrender of Yorktown interactive
The full exhibit website can be found here. The American Revolution: A World War