Interactive Storytelling, AR
Muse is a short film created for the Big Screens class at ITP. Designed for the 120 x 11 foot video wall at the IAC building, Muse also incorporates an augmented reality component to further immerse users into the film.
The IAC building at 527 West 18th Street has a very large video wall.
120 feet wide and 11 feet tall to be exact.
Our team spent many days in this space. Feeling and hearing it. Your sense of time and perspective changes when you're in here looking at that giant screen. Because the space was so large, we felt feelings of loneliness and loss.
We focused in on these emotions and came up with a fairytale of sorts.
It was difficult to get used to storyboarding for such a wide width. And because there are also three pillars, we also considered how the screen and set pieces needed to be broken up.
We used this overlay that contains the correct scale of the video wall and the pillars.
The design of our character came to be when we decided that it should be an androgynous being.
We wanted the character's setting to bring forth the emotional state of the character. There are three major set pieces in the narrative. Each one represents what the character is feeling.
Because we wanted to have complex animation, we decided on creating a 3D rigged model of our characters. This way we could use motion capture to record the character's movements rather than to animate manually.
We also created a 3D model of one of main settings, the City. We wanted the City to feel labyrinthine. By creating a 3D render of the setting, we are able to easily choose what camera angles we'd need. After getting the camera angle and character animation, the setting was then touched up in Photoshop and then both are composited into AfterEffects.
The reason we decided to include an augmented reality component into our piece is because we wanted to introduce this motif of masks. And how we use these masks to both enhance and reduce experiences.
Users will be able to use the AR app to see more content, however, they're also missing the grandeur of the giant screen. This relationship between device and environment was important for us.
We also were intrigued by the idea of introducing Small Screens into Big Screens.
We used Unity3D with the ARKit and ARCore plugins to create an audio reactive app that enhances the visuals of the main content on the big screen.
And after weeks and weeks...
of redos and rendering
and recuts and more rendering...
The premiere was a big success. All the work my team and I did in these past two months resulted in something I felt precious to me. I am proud of my team and of everyone else who presented that night.
It was a terrifying and sleepless two months,
and I'd do it all over again instantly.
Special thanks to
the Big Screens class,
and Ji Young Chun