Self Portrait: Phonetic Keyboard

When I was a teenager, I always found myself much better at talking to people over online instant messengers like AOL Instant Messenger than in real life. In real life, I was awkward. Online, I was a George Clooney. 

I decided to create a version of me that speaks whatever I typed. I recorded a short video of me saying the phonetic sounds of each letter of the alphabet. I then associated each alphabetic key on the keyboard to the specific timestamp of when I said the corresponding letter. 

It ended up being more of a beatbox soundboard thing, but if you type at the right cadence, it sounds kind of like English.

demo: https://bryanjhsu.github.io/phonetic-keyboard/

code: https://github.com/bryanjhsu/phonetic-keyboard

Synchronous Technologies: Youtube shared and Showgoers

YouTube has recently released a chat app to watch videos with your friends. I've tried to adopt this chat app and tried to force it upon some of my closer friends. Its usage lasted for a whopping two days. The purpose of the application was so that you could watch Youtube videos while chatting about with your friends. However, there were a bunch of disappointing features of the app

You're not really watching the video with them. I expected some sort of live control, where the video would have shared playhead control access among all the users. If we were all watching the same content at the same time, we would feel connected. It would feel like I'm hanging out with my buddies at someones house. Without this feature, this app no longer becomes special because it loses that "live" feature. Also, the keyboard covers up 50% of the screen real estate, covering up all of the chat. This drives the unconnectedness of this application even more.

While I was in a long distance relationship with my girlfriend, we used this app called Showgoers. It was, among many others, a Google Chrome extension that allowed multiple users to sync and watch a Netflix program. If I paused the movie, the movie would pause at the same spot on her screen as well. We were connected by this strange limitation. We used this app along with Skype so that we could see each others faces in the corner of the screen while watching a movie. This was an example of a live technology that actually felt "live". It was like I was watching the movie right next to her.

I feel that "live" technologies require interaction with other users. If something is live, you need to be able to affect others that are using the same technology as you. If I told my long distance girlfriend to watch a movie by pressing the play button on our separate Netflix accounts, I could easily in the middle of the movie pause the movie to go the bathroom and she would never know. There needs to be something that connects the two users so that we can both see and hear each others reactions as they are happening.

Bob Ross Reads Your Fortune

For my final, I am tweaking the tarot card idea I had in a previous assignment. I found that using a movie as the source material for a deck was not always effective. They were also too long and certain scenes out of context left very little to interpret for the reader. Then one night as I was browsing Netflix, I saw a recommended series for me: Bob Ross's The Joy of Painting.

I've known about this series for a long time, but only as a sort of internet meme. I've never actually watched an episode of it before. So I watched an episode, and it was hypnotizing. I knew that I found the solution to my problem and the solution was Bob Ross.

How it works

The "deck" is an episode of "The Joy of Painting". I "shuffle and deal" the deck by choosing five random numbers as timestamps, and each of these numbers are assigned to one position of the spread. In this instance, I am using a Five Card Spread. When a card is selected, a clip of the video at that particular timestamp will play for a small amount of time, and loop until another card is selected.

Interpretation

As for the interpretation of these cards, there were several factors to consider:

  • Bob Ross is constantly talking in his videos. And everything he says can be seen as a metaphor for life, which helps the interpretation of the reading. 
  • The beauty of a Bob Ross video is that you can always determine the current state of the painting. This kind of grounds the video into a very clear and discrete sense of linearity.
  • And there is of course a lot of color in the video. So color is one more factor that can be used for interpretation. http://www.color-wheel-pro.com/color-meaning.html

Bob Ross is also incredibly positive, which is what I needed now as the semester is closing and the finals are coming .

code here

app here

Note that any video can be used as the source as long as it's in the same folder as all the other files and it's named bobross.mp4

 

reddit fortunetelling

As I browse reddit, I would often automatically assume the subreddit that each post belongs to. I am usually correct with my guess. Inspired by the Zener cards,  I wrote a simple chrome extension that removes the subreddit from the post on reddit posts. I feel that this removes some sort of subconscious bias I might have before clicking on these posts. It also creates a surprise factor for whenever I click on a post, causing a combination of fear and excitement.

var subs = document.getElementsByClassName('class="subreddit hover may-blank"');
//subs on list page

for(var i = subs.length - 1; i >= 0; i--)
{
    subs[i].parentNode.removeChild(subs[i]);
}

I think because I am so accustomed to the subreddits that I am subscribed to, it isn't too difficult for me to accurately guess the correct parent subreddits. However, I think that if I were to do the same experiment with someone else's account, it may lead to more surprising results. I also think that a similar project could be down with the top comment for each reddit post. reddit has a very hivemindy thought process, so it is often easy to predict the top comment of a post. 

Elegy

I started to analyze my Google Hangouts history in some of my previous classes. I thought that analyzing conversation history could be a pseudo-natural event that could be turned into a fortune. Although we are always in control of what we type, I believe that when we instant message with those we are closest to, our fingers move without thought. 

By analyzing the frequency of messages between myself and two of my ex-girlfriends, I was able to create these songs that represent the unbalance in our relationships.

For every week of texts, I counted the number of characters that I sent and the number of characters that she sent. If I sent more characters, than that week would be represented by a single random guitar note. If she sent more characters, that week would be represented by a single random piano note.

pseudo-code tl;dr:

every note = 1 week of texts

me = 🎸, her = 🎹

if me > her, play 🎸, else play 🎹.

Here is the song that is generated with the messages between me and my most recent ex-girlfriend, GC.

Here is the song generated with my ex before GC.

If you couldn’t tell, the breakup between JJ and I affected me a lot more than that of GC. Both of these songs kind of predict why these relationships ended due to the unbalanced nature of the conversations had.

I decided to sonify the data because I think being able to listen to a whole relationship condensed into a single elegy was more melancholic and romantic.

The majority of the code written lied within parsing the CSV document containing all of the message between myself and the girlfriend. This can be seen here:

https://gist.github.com/bryanjhsu/cfbfdddd073e695acc5683a828154b96

Whitney Visit

My visit to the Whitney was an interesting one. I slowly realized that I started to analyze the users of each piece much more than the piece itself.

Larry Bell, Pacific Red II (2017)

It seemed like most of the viewers of the piece were taking selfies in the reflection of the boxes more than actually viewing the cityscape through the red glass, which may or may not be intentional. I think it might because of how the inner cube made the reflection of the user stronger, so as the user walks past the cubes, they first see the city, and then their own reflection.  Since the cubes both act as a filter as well as a mirror, it's a very nice piece for Instagram.

Porpentine Charity Heartscape

I only played one of the choose your adventure stations in this room, but I found that it disturbed me much more than the real violence piece. The juxtaposition of how the story starts and how it ends greatly affected me. Because we associate these choose your adventure games with childhood and fun, when the game takes a turn into the weird, I felt a sensation of fear.

Throughout the piece, the text was aligned left and stopped about a third into the screen.There was a moment when the text started to go off the screen with text that seemed nonsensical. I like how the artist breaks rules that were previously established to stir up an emotional response in the user. 

John Rippenhoff's "The John Rippenhoff Experience"

The Rippenhoff Experience was an interesting attraction. There was always a line for it due the nature of its setup. I think that the sight of seeing someone sticking their head into something mysterious helped with that, inciting others to stand in line and wonder what is actually inside this mystery box. The concept of hiding an infinity inside a tiny box was very strong with the piece. I liked how you needed to ascend the ladder in order to enter to this space, acting as a kind of stepladder to the next dimension or something. I'm unsure if it was intended or not, but the inside of the box was very warm. It seemed to add to the experience since the different climate inside the box added to the fact that the user is entering a totally new world.

Jordan Wolfson’s “Real Violence”

Real Violence was also a big attraction, being the only VR installation in the museum. Not unlike the John Rippenhoff experience, much of the intrigue was watching others interact with the piece. While I waited on line, I saw some people take off their headset immediately, some until the end of the piece. People left with disgust and laughter and silence. The commentary after was also entertaining. "I need to go watch some Spongebob now."

I decided to watch the whole thing through. It was a bit blurry since I had to take my glasses off. I had to really squint to see the violence, which might have added to the experience. I squinted the entire time. When the audio cut off and then the video, I took off the headset and saw that I was the last person still at the exhibit. I wasn't sure whether or not to wear this accomplishment proudly or shamefully.

Barkley L. Hendricks’ “Steve”.

It reminded me of Robert Irwin's work, on how objects blend into the background. The subject seems to almost disappear into the canvas, yet he doesn't because he looks so damn cool. The reflection off  of Steve's sunglasses strike me the most. The reflection looks like he's inside the interior of a church, looking at stained glass. However from our perspective, his backdrop is a pure white. Words that popped into my head were heaven, angel, style, swagger.

Justification of Putting the Audience Through a Difficult Evening

We view the past as a series of choices. And because we have the knowledge, we often know which choices are good and which are bad. We have the answer key and we look at our past mistakes with certainty.

However, when we're in the moment, the answers are unclear. We answer blindly, not because we didn't care, but because we were under pressure. I remember taking the SATs in high school. When the proctor started to countdown to the final seconds, the questions on my page stopped being important. Only the answers mattered, no matter right or wrong.

I think that it's important for people to be constantly taking these exams that test us because it's the best practice we could ask for. We need to practice with art, plays, film, and stories. By experiencing these mediums, we are transported to worlds where we could choose right or wrong without major consequence. And when we see the outcome of those choices made, we have more knowledge of what to do in the future. We need the practice because when the real exam comes, we need to be able to answer those important questions without hesitation.

Social Editing

I created a tool that allows users to collaborate and edit a shared video with a preexisting source. I was inspired by SoundCloud's commenting system in which users are able to comment at specific moments in a song. Sometimes videos are simply too long and you only want to see the good parts. This tool allows you to mark your favorite moments. After a moment is marked, it will play in the communal video for a short amount of time. 

For more OSC posts, go to https://itp.nyu.edu/classes/osc17/author/bjh315/

a filmic Oracle Deck

What's your favorite movie? Or a movie you've seen recently.

Movies generally follow a three act structure. In the first Act, there is an INCITING INCIDENT. Along with a PROTAGONIST and an ANTAGONIST

This leads to act two, the CONFRONTATION.

Finally in act three, the CLIMAX and the RESOLUTION.

I wanted to turn any movie into an oracle deck. Instead of physical cards, the deck would include short scenes from a movie. The positioning of these cards would indicate the five main themes of a narrative:

1. INCITING INCIDENT - the current situation 

2. PROTAGONIST - describes your current state

3. ANTAGONIST - describes your troubles

4. CONFRONTATION - what will happen when you face your troubles

5. CLIMAX - what will happen at the highest or lowest point

6. RESOLUTION - your future

I decided to use video and film as a format for an oracle deck because it followed the general rules of cleromancy. A film has a discrete number of frames, which can be combined into a discrete number of scenes. I enjoy how the meaning behind each scene can be interpreted differently. Someone having prior knowledge of the film versus someone without would have a different reaction to the card that they view.

A Seamless Ritual

I order Seamless quite often. Especially when I have all of these $10 off $15 coupons. Although these coupons say they are for specific restaurants, they're actually usable for anywhere. A sandwich shop near my apartment gives stacks of these coupons freely.

However, these coupons only work on "new" accounts. The creation of these accounts is a special ritual for me. The process of getting a great discount and delicious food excites me like no other. The fact that I have to hit a $15 dollar minimum is great. It lets me order things I normally would not, sometimes getting a little fancy. 

One negative is that this is not healthy for me. I used to be unable to finish $15 of Seamless delivery, but lately I've been completely finishing each meal. I also realize that this is not a good thing I'm doing. I am cheating a flaw in Seamless's system. So to feel better about myself, I always give a generous tip to the delivery person. But not too generous since I still want to save money. I'm a bastard.

My inner bot

The Real Shooby is a Twitter bot that mimics the messaging style of myself when I talk to my close friends. It analyzes over 150,000 messages that I have sent to my friend and uses a the Rita.js library to generate markov chain sentences. And through the use of the Twitter API, it can talk to any user who messages it. The majority of the time on this project was spent on working with large data sets. I definitely grew to be more comfortable working with them after this project.

I use Google Hangouts as my main messaging application when talking with my closest friends. Google provides a service called Takeout that allows any user to download their information, which includes their entire Hangouts history in a JSON format. My Hangouts JSON file ended up to be approximately 550mb (2 million lines), which was much more than what I expected.

The JSON file was a bit obtuse. And because it was so big, it was difficult for me to properly analyze it so that I could parse it myself. So I decided to find a tool to parse the JSON for me. The one I ended up using was this one: https://hangoutparser.jay2k1.com/. The problem with this tool is that it only accepts JSON files 500mb or less which meant that I needed to split my current JSON into smaller file sizes. This took a bit of time since I needed to understand the JSON’s data structure in order to not break the file.

????

????

After the tool parsed the JSON, it allowed me to download CSV files of each conversation. The ones that interested me were conversations that had the most messages sent which included my close friends and my ex-girlfriends. These CSV files ended up not being totally perfect. There were many things that disrupted the structure such as carriage returns, URLs, and weird emoticons that we liked to use. So I had to clean up these files as well, which led to a little bit of information lost.

I then used a csv-parse library to parse these CSVs. Because this project mainly focuses on my own messaging style, I only had to extract messages that I sent from these conversations. After compiling all of my messages, I noticed that my messaging style is not very conducive to analyze as complete thoughts. I like to message rapidly

in a sort of

stream of consciousness,

kind

of like

this.

So I also needed a way to determine complete thoughts. I ended up appending messages that I sent within 5 seconds of my last message. This was done by finding the difference of time in the timestamp of consecutive messages.

After I was satisfied with the final text file, I used the RiTa library to analyze the text with markov chains and generate sentences. The result was bizarre and familiar.

My main code Bot.js on digitalocean server, run with forever lib

https://gist.github.com/bryanjhsu/4353b3cbb14dc6c2163db88148018d2f

For now, everytime someone DMs the bot, bot generates a sentence from markov chain and replies to the sender. The bot also refollows any user who follows it, allowing the user to easily DM to the bot after following.

I am definitely going to continue working on this bot. I would like to have the bot recognize key words in received messages and reply in a way so that it becomes more "conversational".

Bonus: here is a conversation my ShoobyBot had with Liarbot (a bot that tweets anything you DM it)

Brunelleschi Brainstorming

For the midterm project,  we (Jina, Jixuan, Mona, and I ) wanted to create the illusion of depth and height on flat surfaces.

In room 50 of ITP, there is a grid of square panels. By projecting vanishing points on each square, we could play around with a character or object climbing or falling into these squares. 

As we were looking up and down the stair well of the building, we noticed that the underside of the stairs still look like stairs, creating a sort of upside down world. There are also two very distant vanishing points as you look up and down the center of the well. We can use these properties to create a sense of vertigo and perplexity.

On the fifth floor, the top and bottom of a wall is color separated. We found that viewing a corner of this wall from a low perspective makes the corner look concave, as if the corner is going inwards. Switching to a high perspective makes it easier to view it as a convex object. If we project a similar image onto a blank wall, we could create an illusion of an object being both inside and outside depending on the perspective of the viewer.

Talking Directly to the Elephant

To talk directly to the elephant I focused on laugh tracks. It reminded me of this video that somebody made to make fun of the TV show, The Big Bang Theory. Somebody took out the laugh track from one of its episodes and it turned it into an incredibly awkward stage play with terrible comedic timings.

Due to my hatred for the television show "The Big Bang Theory," I expose how unfunny the show actually is when you take out the laugh track. Enjoy! I DO NOT OWN THE BIG BANG THEORY, PLEASE DON'T SUE ME. This is meant as a joke.

Although the video is funny, I don't quite find it fair to judge a show's comedy solely based off this experiment. TBBT writes and edits the show with the laugh track in mind, so taking it out would be taking an integral piece out of a machine. I personally do not like TBBT, but I do not mind having a laugh track. One of the main reasons I enjoy going to the theaters is to experience a film within a large group of people. The social aspect of watching film or theater is incredible to me, so a laugh track is a crude yet effective way to simulate such an experience.

Most modern shows no longer utilize laugh tracks. It would be an interesting experiment to do the opposite of the video above by adding in laughs to a show without a laugh track. Although it would obviously disrupt the original intent of the show, much like taking away the laugh track from TBBT, it would be interesting to see if it actually adds or detracts from the show. 

 

Liarbot

If the bot is followed, the bot will follow the user back and DMs them a message:

If the user responds then the bot will take the user's message and tweet it to the public. 

View the code here.

Exposing ROY

I wanted to find a correlation between a person's heart rate and the moment when they find something funny. I used a heart beat sensor to detect my own heart beats while I watched a recent episode of It's Always Sunny In Philadelphia, a show that will often make me laugh aloud. In addition to measuring my heart rate, I also jotted down the time stamp of the show whenever I found something funny or shocking, so not necessarily whenever I'm "laughing out loud".

I found that the heart beat rate seems to only change when I am audibly laughing. There were several moments in the episode when something funny happened, but there was barely a change in the IBI. There were also many shocking moments in the episode where I think I audibly gasped, but that also did not seem to affect the heart rate enough to be noticeable. The IBI also seemed fairly consistent, going down to about 300 ms whenever I laughed. The highest my heart rate got to was about 105 bpm.

After my test results, my conclusion is that the heart beat seems to change only during more physical reactions. I hoped to find a more clear correlation between the actual moment of someone finding something funny and the heart, but it seems like I might need to use a different sensor to detect that. I also wish to do the same test with media that are less comedic but still bring up emotional responses from the viewer.

Node.js Twitterbot

I used a JSON file that contained many music genres and appended the genre to the end of a Youtube search URL. This allows the user to find new genres of music that they might not have heard.

Cheap Twitterbots

The first Twitterbot assignment was to create five bots with the Cheap Bots Done Quick tool. Here are the five bots I created. Each description links to the JSON used to create the bot.

Combines one line from Shakespeare's sonnets and one line from Eminem's Lose Yourself.

Almost all of the data I used were from Darius Kazemi's Corpora Project, except for the Eminem lines. I found it interesting that all of these bots were not ideated until after going through all of the different categories of data in the Corpora Project. I would browse through these categories and keep ones that had potential in my mind. Tracery was fun to use, but I wasn't how to use JSON values nested within objects (for example something like this). This would open up a lot more possibilities to use pre-existing JSON files.

I was surprised at how poetic some of these bots could be, even when I had no intention of creating something of that nature. My favorite is probably still the JuliaChilds one since it was the one I refreshed the most to see what bizarre thing the bot would create.

 

Theory

Self-awareness and self-discipline are both areas of interest for me. The first chapter of Stumbling On Happiness was a great read and expanded on both of these topics. Too many times have I done something so obviously detrimental to myself. I want to be able to control the disobedient elephant in my brain.

I'm not too sure what theory this would fall under, but I will definitely be reading more of Stumbling On Happiness. I also intend to check out the other books under the Illusion of Rationality category on the website.