Interspecial
An interactive installation with complex logic and a rich multimedia interface.
Customer:
G.Eco – Associazione Culturale
Challenge:
Create a rich multimedia interface that interacts with kids through objects with embedded NFC tags and big colored pushbuttons in an adventure to find out the nature of four planets through quizzes and videos.
Year:
2022
A returning customer
G.Eco has gained some relevance in Italian science events for youngsters thanks to their highly creative interactive installations. I had the honour to work with the team in several occasions and this last assignment was a real challenge for three main reasons: the complexity of the design, the very limited time available to deliver and, last but not least, the need to execute everything as a remote service.
After a quick brief with the G.Eco team we agreed on an hardware configuration based on a computer, a big screen, an Arduino with NFC reader, some LEDS and an IKEA piece of furniture to be laminated with vinyl print.
On the software side, after some analysis, I decided to rely on Processing for all the User Interface, based on many graphics, videos and still pictures grouped in sequences of interactive quizzes. Processing can interact easily with Arduino through standard serial communication and therefore all the physical interactions were managed at the Arduino Level.
Processing vs Arduino
I am fluent with Arduino programming and switching to Processing wasn’t a big issue for the syntax, but the way the code is executed is a true nightmare if you are used to programs that execute going through conditions and delays waiting for events to happen. With Processing you have a main loop named draw(), which is continuously executed doing things like animations and video playbacks until something happens, like a mouse interaction, a key press or a serial input. In this case, the execution jumps to the routine defined for the event and then the execution goes back to the loop.
As an example, the playback of a video is achieved loading a frame in a buffer and showing it as a still, then the pointer to the frame is incremented and the loop is repeated. This is executed at the standard framerate and therefore the “video playback” is done.
In this Interspecial application it was a continuous flow of videos, still images, inputs from NFC or buttons as well as lighting the right LEDS under the pushbuttons with a bidirectional flow of data with Arduino.
The different screens / quizzes were twelve and ten other screens were used to guide the player into the exploration.
The code on Arduino was straightforward and took around five hundred lines to manage everything. On Processing, the code was much more complex with over 2400 lines.
How we made it
The time given for development was just two weeks, including the sourcing of parts and the creation of the unit in Italy mirroring the prototype made in Dubai. The Italian team was skilled, but on the electronics side, they had little experience. Because of this, I prepared a solution that was almost solderless and based on snap contacts and Dupont connectors.
The first step was to allow the Italian Team to get the components required to put together the whole system, without the final design, but fully functional for testing. The second step was the definition of the documents necessary to define the complex logic of the twelve different interactions and a shared Google sheet was the chosen solution. A specific shared folder on Google Drive was the place where videos, images, code and anything else were stored.
At the end of the first week, the hardware was ready on both sides; on my side, it was faster because I had all the parts in my lab. We decided to develop graphics in parallel with the code and therefore we agreed on placeholders with precise names and sizes, so that all the real graphics could be integrated with a simple file replacement and no change in the code.
The whole interactive experience was based on a set of repeating situations but with different graphics. I was therefore able to prepare the basic set of four interactions, make them fully debugged and then copy and paste for every planned set of graphics. Sadly some buggy libraries and limitations of Processing required some workarounds to playback correctly every video.
Teamwork
A continuous exchange of the code with the Italian Team was essential for testing and debugging, while on their side the creation of the graphics and the console was proceeding as fast as possible. Working over 14 hours a day, I managed to finish the code in five days, leaving enough time for testing, debugging and last-minute changes.
Pictures and video connections were used to help the Italian team in the assembly process.
Prepare for trouble with “sesamini”
The past experiences of G.Eco with kids and such kinds of installations allowed them to require specific “shortcuts” throughout the whole program to speed up the whole gameplay, normally around 30 minutes. Each level of play – the specific quiz, the planet and the whole set of planets – had a special key on the wireless keyboard connected to the pc to mark as completed that part. In this way every possible situation was managed and any slow performance by the kids, or a long waiting list was sped up thanks to these “shortcuts” – friendly named “sesamini” by the team.
A big hit with kids
The Interspecial installation was used in 2022 edition of Festival della Scienza in Genova, Italy and in another festival in September 2023, also in Italy. In both events it was a big hit with kids.