Muse

 

Interactive Storytelling, AR

Role: Co-creator, co-director, editor, animator, developer (AR)

Time to completion: 2.5 months

Team: Chang Gao, Yuqiao Qin

Tools : Unity3D, AfterEffects, Photoshop, Premiere, Blender, MotionBuilder, ARKit

Links : full performancewebsite

Synopsis

Muse is a short film created for the Big Screens class at ITP. Designed for the 120 x 11 foot video wall at the IAC building, Muse also incorporates an augmented reality component to further immerse users into the film. 

 
 

Pre-production

 

The IAC building at 527 West 18th Street has a very large video wall.

iac wall.jpg

120 feet wide and 11 feet tall to be exact.

 
 

Our team spent many days in this space. Feeling and hearing it. Your sense of time and perspective changes when you're in here looking at that giant screen. Because the space was so large, we felt feelings of loneliness and loss.

 

We focused in on these emotions and came up with a fairytale of sorts.

 
 
 

It was difficult to get used to storyboarding for such a wide width. And because there are also three  pillars, we also considered how the screen and set pieces needed to be broken up.

 
IMG_0278.JPG
 
 

We used this overlay that contains the correct scale of the video wall and the pillars.

 

The design of our character came to be when we decided that it should be an androgynous being.

 
 

We wanted the character's setting to bring forth the emotional state of the character. There are three major set pieces in the narrative. Each one represents what the character is feeling.

 
 
trip1.jpeg
 

lost,

floating,

adrift

 

wonder,

fright,

adrenaline

trip3.jpeg
 

fulfillment,

apprehension,

doubt

 

Production

 
 

Because we wanted to have complex animation, we decided on creating a 3D rigged model of our characters. This way we could use motion capture to record the character's movements rather than to animate manually.

 
 
 

We also created a 3D model of one of main settings, the City. We wanted the City to feel labyrinthine. By creating a 3D render of the setting, we are able to easily choose what camera angles we'd need. After getting the camera angle and character animation, the setting was then touched up in Photoshop and then both are composited into AfterEffects.

 
Screen Shot 2018-02-06 at 11.24.57 PM.png
aftereffects.png
 
 

AR

 

 

The reason we decided to include an augmented reality component into our piece is because we wanted to introduce this motif of masks. And how we use these masks to both enhance and reduce experiences.

Users will be able to use the AR app to see more content, however, they're also missing the grandeur of the giant screen. This relationship between device and environment was important for us.

We also were intrigued by the idea of introducing Small Screens into Big Screens.

 

 
unity2.png
 
 

We used Unity3D with the ARKit and ARCore plugins to create an audio reactive app that enhances the visuals of the main content on the big screen.

 

 
unity.png
 

And after weeks and weeks...

of redos and rendering

and recuts and more rendering... 

 
s1.jpg
s21.jpg
 

The Premiere

12/8/2017

 
iac3.png
iac2.png
Screen+Shot+2018-02-01+at+2.10.57+PM.png
0C4A0030.JPG
Screen+Shot+2018-02-11+at+1.28.19+PM.png
Screen+Shot+2018-02-04+at+5.05.27+PM.png
iac6.png
iac7.png
iac8.png
Screen+Shot+2018-02-04+at+5.04.34+PM.png
iac4.png
 
 

The premiere was a big success. All the work my team and I did in these past two months resulted in something I felt precious to me. I am proud of my team and of everyone else who presented that night.

 

 

 
 

It was a terrifying and sleepless two months,

 

and I'd do it all over again instantly.

 

 

 

Special thanks to

Mimi Yin,

Lu Wang,

Alon Chitayat,

Gabe Barcia-Colombo,

Kevin Chen,

the Big Screens class,

m2m,

and Ji Young Chun