Stitching Stories
A non-linear story controlled by a hardware controller
Programmer | Collaborator: Sisa Bueno
Introduction to Physical Computing, Fall 2014
November-December 2014
“Design Question: Can a story become an individual experience by having the audience decide how they experience it?”
Stitching Stories is an interactive story that the user controls using a Bop It toy as a physical controller. The story is comprised of multiple video clips which is controlled by a Processing sketch. The original hardware in the Bop It was removed and replaced with an Arduino Micro; as the user interacts with the Bop It's components, the Arduino passes the information to Processing to change the clip.
I worked on creating the Processing sketch that controlled the video clips being shown to the user and integrated it with the Arduino sketch to read which control was being used so the sketch would show the correct clip.
Stitching Stories was exhibited at the ITP Winter Show 2014 and mentioned in Makezine's coverage of the show
Case Study
Creating an Individual Experience with Film
The fundamental question of the final project brief was: how can we use hardware to create a novel interaction? Sisa pitched the idea of a cube that could be used to produce an interactive film about two characters. Most media is passively experienced, but the cube would give the user the ability to control how they experienced the story by cutting between points of view. At the time, we weren't thinking a lot about the type of user the project would be directed at but we wanted to keep the interaction simple enough that it could reach a broader audience.
The Making Of
We brainstormed on what the experience would look like and what we would need to do to to make it, coming up with a system diagram and a project plan:
We planned to fabricate the controller ourselves while Sisa filmed the videos and I explored how the controller would connect to the video.
As we were doing our technical explorations, we found that some of the technologies were not the right fit for the project or cost too much. We also found that the cube didn’t provide a lot of interaction capability. We brainstormed on alternatives to the cube when I suggested using a Bop It, the party game from the 90’s. What inspired me to think of the Bop It was its playful toggles and interactions, which influenced the overall feeling of the story. For example, tugging on the Pull It control showed the user a clip of our characters playing tug of war.
This created a more playful interaction that connected the hardware controller interaction to the media being played on the screen, and we wanted users to be familiar with the clips they'd see in relation to the Bop It as the interactive video editor as fast as possible during their time with the piece.
Sisa wrote the overarching story and managed the video production side of the project, while I built the program that would play the videos and connect the computer to the Bop It to read in the input from the user. We both soldered the custom Arduino circuit that replaced the one built into our Bop It.
Lessons Learned
This project has evolved so much from the concept that Sisa pitched in class and that I got interested in - it took us down many paths in regards to the technology used to implement it as well as the interaction model (since we changed the controller entirely). There was a lot of stress and pressure, but I think that Sisa and I had a good deal of patience that helped us get this done well and produced a working project.
Documenting all the meeting notes in a shared Google Drive was so helpful because we could both see what was going on and share resources easily. We kept on track pretty well when we were taking notes to have something to go back to.
The residents were an invaluable resource for us; without Xuedi's hardware knowledge and Sam's help with my Processing debugging, we wouldn't have made Stitching Stories as it existed.