EDPX 4010 Project
So as I started coming up with ideas on what kind of tool to create I kept coming across a problem. Every time I would tell someone about my idea they would either question something that I haven’t thought of making the project more complex OR question the purpose of digital tools which I would respond to by thinking of a useful tool that I now want to create. I am going to limit this post to the two main ideas I have been pondering.
The first idea is related to digital tools for music performance. The best tool for improving music performance is and has been sound recording since its inception. When someone performs their auditory feedback can be compromised by the signals they send their brain to perform in the first place. Listening to a recording can allow performers to hear the actual physical sound compared to how they thought it was played. Another useful tool is video recording of performances. However it should be noted that it is most helpful to performers when synched with audio so posture and positioning can be compared to sound output. So as we learn about all the different sensors that can be used I started to realize how useful they could be. Specifically I am looking to design and construct a smart bow for classical string instruments.
The main components of the bow are the stick, hair, and the frog which is the part of the bow that you hold. The hair is glued and set on the tip side and can be adjusted to various tensions with a screw set into the stick on the frog side. I want to 3D print a frog that can contain a specifically designed arduino within it. It would contain an accelerometer to show bow position and speed. I also want to put in a sensor that can show the difference in tension exerted onto the string. I believe it would be most efficient if it contained a rechargeable battery so the musician is not inhibited by any cords. This system would need to be linked to a computer to run the data so I was thinking of a bluetooth function to connect a cell phone. Finally, the data would need to be compiled with an audio recording because just like the video recording, the data can only tell us how to improve if we have the auditory information to compare it to. I want to prototype this with a Bass bow as bass bows are the largest and have the most space to work with in the frog for components. The creation of sound via string vibration is a very dynamic process and I believe that with the creation of smart bows/instruments that we can break down those dynamic components into easily digestible parts allowing for easier music learning and performing.
The idea of a smart bow is very interesting and is certainly a digital tool but I don’t find it satisfying in an artistic way which brings me to my next idea; colorizing music. This seems like it would be simple but it turns out, it ain’t! There is a brain disorder called Synesthesia where perceptual stimuli will be mixed so that someone might experience a sensation in more than one medium. The example that I will be using is people who see sound as color. It is interesting because musicians have long used artistic ideas like color to describe music but there is not any standardized note to color ratio. The little bit of research I have seen shows that it is hard for people to agree on how it would work and while the idea is great it is not as satisfying when put into practice. There are apps that exist that colorize pitch but they are mainly used for tuning purposes.
That brings us to some of the problems. It is not easy to take an auditory signal and break it down into one pitch. Unless the sound is a sine wave it will contain a lot of harmonic content that are each their own pitch as well. This can be confusing for systems trying to analyze the pitches. Tuners are able to do this with one instrument with relative success but that breaks down when you start to get more than one voice. While there is a lot that could be done with one voice, the true test would be to colorize the texture of larger ensembles which contains a cornucopia of harmonic content.
Let’s say there is someone that figures it out and we are able to digitally break chords into their component pitches with a high consistent success rate? That still doesn’t help us in how to represent the content with color. There are certain functions of music theory that are universally experienced that I am not sure how you would represent with anything but sound. If you have ever learned to play guitar you learn pretty quickly that once you know three chords you can play most popular songs. This are the first, the fourth, and the fifth chords of a scale also known as the root, sub-dominant, and dominant chords. When you play the dominant chord followed by the root chord you will hear what is referred to as a cadence. The phenomena is described as resolving from one chord to another and this exists regardless of what key you are in and whether or not it is major or minor. How do you represent that with color? Can colors resolve into each other in a tangible way? Also, a chord by definition needs at least three notes so does that translate into three different colors? Three hues of the same color?
I am positive that the only way to figure out the answer to any of these questions is to experiment with the way they look. Well, you can’t experiment with the way it should look until you see the way it does look if programmed. So I believe I will have to code the music into a processing sketch. This is particularly meta as music is already a code and we would just be creating a simulacra of a simulacra of an auditory symbol. I would approach the programming of music with a mindset on music history. I would follow the technological advancements of music notation from the dark ages until modern music and try to create visual representations to match. This will be easier the earlier the music as the voicings and textures are simpler. As the music gets more complex I will have to find other visual effects to model the auditory data. This class has made me start thinking about biological data as a medium for art and I think that could be an interesting idea to explore. Maybe the video of a cell dividing into two under a microscope could represent a modulation into another key. Whatever the method, this is sure to be a lifelong project with a million iterations.
Life!?! I got two weeks! Baby steps. I want to colorize music so I think I will start using a microphone as a sensor. I would patch the microphone into a program that would analyze the sound in two separate ways, amplitude and pitch. I will transcode those into brightness and color respectively using LED lights. Brightness would be directly correlated to amplitude of sound. The pitches would be broken down into 12 different colors based on the 12 pitches of a chromatic scale. The colors of same notes at different octaves would have slightly different hues displaying lighter hue the higher the pitch. Once I can get this working I would like to build an infinity practice mirror. Musicians will practice something until it is correct. Then you have to practice it correct 10 more times! Or just once in my infinity practice mirror!