EDPX 4010

EDPX 4010 Project

So as I started coming up with ideas on what kind of tool to create I kept coming across a problem. Every time I would tell someone about my idea they would either question something that I haven’t thought of making the project more complex OR question the purpose of digital tools which I would respond to by thinking of a useful tool that I now want to create. I am going to limit this post to the two main ideas I have been pondering.

The first idea is related to digital tools for music performance. The best tool for improving music performance is and has been sound recording since its inception. When someone performs their auditory feedback can be compromised by the signals they send their brain to perform in the first place. Listening to a recording can allow performers to hear the actual physical sound compared to how they thought it was played. Another useful tool is video recording of performances. However it should be noted that it is most helpful to performers when synched with audio so posture and positioning can be compared to sound output. So as we learn about all the different sensors that can be used I started to realize how useful they could be. Specifically I am looking to design and construct a smart bow for classical string instruments.

The main components of the bow are the stick, hair, and the frog which is the part of the bow that you hold. The hair is glued and set on the tip side and can be adjusted to various tensions with a screw set into the stick on the frog side. I want to 3D print a frog that can contain a specifically designed arduino within it. It would contain an accelerometer to show bow position and speed. I also want to put in a sensor that can show the difference in tension exerted onto the string. I believe it would be most efficient if it contained a rechargeable battery so the musician is not inhibited by any cords. This system would need to be linked to a computer to run the data so I was thinking of a bluetooth function to connect a cell phone. Finally, the data would need to be compiled with an audio recording because just like the video recording, the data can only tell us how to improve if we have the auditory information to compare it to. I want to prototype this with a Bass bow as bass bows are the largest and have the most space to work with in the frog for components. The creation of sound via string vibration is a very dynamic process and I believe that with the creation of smart bows/instruments that we can break down those dynamic components into easily digestible parts allowing for easier music learning and performing.

The idea of a smart bow is very interesting and is certainly a digital tool but I don’t find it satisfying in an artistic way which brings me to my next idea; colorizing music. This seems like it would be simple but it turns out, it ain’t! There is a brain disorder called Synesthesia where perceptual stimuli will be mixed so that someone might experience a sensation in more than one medium. The example that I will be using is people who see sound as color. It is interesting because musicians have long used artistic ideas like color to describe music but there is not any standardized note to color ratio. The little bit of research I have seen shows that it is hard for people to agree on how it would work and while the idea is great it is not as satisfying when put into practice. There are apps that exist that colorize pitch but they are mainly used for tuning purposes.

That brings us to some of the problems. It is not easy to take an auditory signal and break it down into one pitch. Unless the sound is a sine wave it will contain a lot of harmonic content that are each their own pitch as well. This can be confusing for systems trying to analyze the pitches. Tuners are able to do this with one instrument with relative success but that breaks down when you start to get more than one voice. While there is a lot that could be done with one voice, the true test would be to colorize the texture of larger ensembles which contains a cornucopia of harmonic content.

Let’s say there is someone that figures it out and we are able to digitally break chords into their component pitches with a high consistent success rate? That still doesn’t help us in how to represent the content with color. There are certain functions of music theory that are universally experienced that I am not sure how you would represent with anything but sound. If you have ever learned to play guitar you learn pretty quickly that once you know three chords you can play most popular songs. This are the first, the fourth, and the fifth chords of a scale also known as the root, sub-dominant, and dominant chords. When you play the dominant chord followed by the root chord you will hear what is referred to as a cadence. The phenomena is described as resolving from one chord to another and this exists regardless of what key you are in and whether or not it is major or minor. How do you represent that with color? Can colors resolve into each other in a tangible way? Also, a chord by definition needs at least three notes so does that translate into three different colors? Three hues of the same color?

I am positive that the only way to figure out the answer to any of these questions is to experiment with the way they look. Well, you can’t experiment with the way it should look until you see the way it does look if programmed. So I believe I will have to code the music into a processing sketch. This is particularly meta as music is already a code and we would just be creating a simulacra of a simulacra of an auditory symbol. I would approach the programming of music with a mindset on music history. I would follow the technological advancements of music notation from the dark ages until modern music and try to create visual representations to match. This will be easier the earlier the music as the voicings and textures are simpler. As the music gets more complex I will have to find other visual effects to model the auditory data. This class has made me start thinking about biological data as a medium for art and I think that could be an interesting idea to explore. Maybe the video of a cell dividing into two under a microscope could represent a modulation into another key. Whatever the method, this is sure to be a lifelong project with a million iterations.

Life!?! I got two weeks! Baby steps. I want to colorize music so I think I will start using a microphone as a sensor. I would patch the microphone into a program that would analyze the sound in two separate ways, amplitude and pitch. I will transcode those into brightness and color respectively using LED lights. Brightness would be directly correlated to amplitude of sound. The pitches would be broken down into 12 different colors based on the 12 pitches of a chromatic scale. The colors of same notes at different octaves would have slightly different hues displaying lighter hue the higher the pitch. Once I can get this working I would like to build an infinity practice mirror. Musicians will practice something until it is correct. Then you have to practice it correct 10 more times! Or just once in my infinity practice mirror!

Disruptive Technology : Telepathy

Telepathy is based off the greek words tele and pathos which roughly translate to distant experience. Merriam Webster defines is as communication from one mind to another by extrasensory means. The term is believed to be coined in the late 19th century by the Society for Psychical Research. The term mainly has the same associations currently but it is starting to go in another direction with tech companies developing telepathic technologies.

Facebook and Neuralink are two companies that have created positions that are believed to be related to telepathic communications. Facebook has a secret division that is developing “brain computer interface technology” that was filling positions a year and a half ago. They would create an interface that would simulate interaction with haptic technology. It is totally believable that Facebook has the means to create technology like this because of their fiscal ability. Their whole business model claims to be to connecting and networking but there is such a possibility of data exploitation that I would never want to use any product they created.

Neuralink is a company that is being financed by Elon Musk. When you look at the different companies related to Elon, they all point to finding solutions to problems of planet exploration. So creating telepathic technologies could be a solution to long distance communication. More than likely the technology would be used to interact with the other technologies that Elon is creating most likely through human computer interfaces as well. Neuralink has expressed interest in implanting chips directly into the brain to enable these interactions. However the tangibility of this kind of device being implanted into humans in the near future is unrealistic. The realities for safety testing indicate these devices would take many years to be approved.

I picked this subject because like many people, I have experienced telepathic phenomena for which I have no explanation. The first experience is one that happens on a regular basis. It is experienced by almost everyone but is reported most commonly with musicians. There is instances while performing or listening to music when individuals will feel a “chill.” This is different from a “chill” that is scary or unnerving. The first time I experienced this was listening to an orchestra live and this is the scenario with which I experience it most frequently. However, as I learned to communicate with colleagues and found that I could experience this feeling when successfully explaining an idea or concept.

The second occurred while a freshman at ASU in the music school. It was getting close to the end of the first semester and I had just bonded with a fellow musician over our appreciation of Bjork. We decided to head back to my dorm to listen to Bjork. While we socializing my new friend spoke about his problem of having too many ASU “points” (school money), that if not used by the end of the semester, is lost. This made me think of a sign I saw at the Union grocery selling cases of red bull for points. As I was about to ask my friend if I could use his points for Red bull he interrupted me and said “Keith, are you going to ask if you can use my points for Red Bull?” I didn’t know what to say. Sheepishly I said yeah as I thought about how he knew. I asked how he knew and he replied he didn’t know how, he just did.

I started doing “experiments” to try and communicate with people for fun. My favorite “experiment” was to try to create “links” with people I didn’t know. Have you ever felt you were being watched? There was a balcony on there third floor of the Music building that most people didn’t know existed. On practice breaks I would go out and watch people walking to and from class. I would consider a “link” a success if the person looked around in a manner different from just walking to their destination. I had the most success under certain circumstances. The people walking were alone and not talking on the phone, and I would have to think about how they looked and moved. The more dressed up a person seemed to help as well. The people who “linked” always looked in my direction but rarely looked up.

So I want to break these phenomena down into how I understand them. I believe the first has to do with frequency and specifically sympathetic vibration. Sympathetic vibration occurs when a sound or vibration is equal to a vibrator/space’s resonant frequency. Resonant frequency refers to the pitch at which something naturally vibrates. Let us pretend that I am playing a Bach prelude. My brain starts firing to create movement that creates sound which creates a feedback loop in the brain. I am simultaneously receiving the auditory stimuli while creating impulses to keep the music continuing. The music is communicated through sound waves that will impact all listeners the same including the performer. This creates similar brain patterns between the collective listeners that creates a “sympathetic vibration” of brain patterns and I believe the physical result is the “chills.”

The second becomes a little more complicated. Our brains evolved to hold images in sequences. Our hunter gatherer ancestors were able to thrive because of these adaptions. Instead of just experiencing a backdrop, they were able to hold an image from that backdrop that is separate from their experience to help guide their decisions. So the first thing we have to do is accept that thoughts of images are physical things. Even if there is no current way to understand how they exist as physical things, we still have to accept them as such if we believe they are being transmitted or transferred. I had a thought of a red bull sign that was transmitted to my friend. I believe it was shared through sympathetic thought processes. However, there was more going on. We were talking about unused “points” and I transmitted an image of a sign advertising red bull for points. He received this image and recognized that it was not his thought. He inferred the context with which it existed (he needed to get rid of points) and found me as the only other source for which this idea could have been created. So while I didn’t transfer the whole thought “I could use points to buy Red Bull”, he was able to interpret that message based on the image.

I believe this also can explain my experiments as they had the most success when I focused on a person’s image. People have a strong self-image and I know I personally become more aware when I am dressed up. So I am creating a brain pattern signal that they are receiving sympathetically. In addition to receiving this signal, they were able to deduce the position from which the signal originated.

Based on these phenomena I believe that humans have an ability to transfer thoughts telepathically. I would create technology to augment these abilities and allow for communication without being physically intrusive. The first version of this technology would look like earwarmers that can connect to a computer. The band across the top of the cranium would contain sensors to monitor brain activity. The ear warmer would contain a “smart rubber band”surrounding the ear creating contact with bone around the base of the outer ear. This band will create unique high frequency audio signals that will be transmitted through bone conduction straight to the cochlea. This signal will be perceived by the brain but not as sound. The technology I create would have a simple interface that you could control with a computer. When two or more people are wearing the bands they would sync their computers to sync the same signal. The bands would monitor the brain waves and give continuous feedback to the signal to match between the users. The users would then communicate images through telepathy.

The companies that are pushing this technology are looking to create an interface with which we can communicate telepathically. My technology would be much simpler and only create audio signals to try and match brain patterns. By eliminating a virtual haptic interface I believe I can sidestep most ethical issues that exist. When we look at the AI technology that is currently being used, we find it prevalent and helpful in everyday life. Search engines use artificial intelligence tools to make suggestions they believe will help our searches is just one example. This technology would be vital for telepathic technology to be effective. Take my example from college, I didn’t transfer the idea but rather an image that enabled my friend to correctly guess the idea that existed.I can find these tools very helpful, but sometimes I feel that it does not help. Either way the separation between the tool and my conscious mind is clear. What happens when we combine the interface to the tool with which we judge the interface to be real. If I plug these tools into my brain would there be away to differentiate between the Artificial Intelligence and the Real Intelligence? I would have a hard time trusting that interface if it is operated by Facebook. They already use the network interface to collect data but at least it is data that we willingly provide. Creating a channel for them to directly access our thoughts would allow them more access than I am willing to give.

The ideas from Elon Musk’s company are even more frightening as they talk about implanting chips directly into our brain that would allow us to interact with computers with our minds. From an ethical standpoint where does the connection stop? Can the user block the chip from accessing the brain? People’s psyches are already extremely fragile. What if you could start uploading memories of alternate realities? This could be the greatest tool of all time for a Fascist government. From a purely technological and science fiction perspective this is probably as exciting as it is frightening.

I just got my neural implant. My brain and body haven’t rejected the implant so today we get to try the first update. I just got back from my optometrist and my prescription hasn’t changed, still legally blind in one eye and poor vision in the other. I take this to my new neural process doctor and they upload a vision correction program specific to my prescription onto my neural chip. I now have perfect vision! My doctor shows me how to turn this filter on in my interface and asks me if I would like to upgrade and add night vision or infra red but I decline when I find out it is not covered by insurance. The doctor asks if there is anything else I need done so I mention that I have been having a hard time losing weight. “No Problem” says the Neural doc, “I can set your metabolism to a higher rate only to recede when you reach your optimal beach weight! You wan’t bigger muscles too?” One year later I had forgotten what my life was like but was forced to remember when my neural chip failed. I didn’t have my glasses and I thought that was the least of my worries. It wasn’t until day 2 that I realized my brain had become reliant upon the neural chip providing impulses…

https://www.technologyreview.com/s/604254/with-neuralink-elon-musk-promises-human-to-human-telepathy-dont-believe-it/

https://www.businessinsider.com/facebooks-building-8-working-on-brain-computer-communication-platform-2017-1

My classmate, Emilie Trice and I were experimenting with cameras and editing video of some of my bass playing. This is my first experiment with it and I love the vivid picture but I wasn’t completely happy with how I cut it.

This is my blog for my Emergent Digital Practices class. I will be posting projects and ideas from my class on this blog.