Final Project: The Raga Machine

The Raga Machine:

The final project for this class was one of the most fulfilling projects that I have worked on all semester. Having undergone 10 years of training in Carnatic music and having sorely missed practicing it for the last three years, this project was a fun way to reconnect with it. Based on Carnatic ragas, my project  was designed to be an eight-key keyboard that could play a variety of ragas (five, in this case). A raga, for context, as a particular combination of notes that songs are composed in. For example, if raga A contains the notes Do-Re-Mi and Mi-Do-Re, then a song composed in raga A will only contain those notes and in that order. I think the Western equivalent of this is a key, but I’m not sure. Ragas are generally divided into two kinds: Melakartha ragas, and Janya ragas. Essentially, Melakartha ragas contain all eight notes that can be sung in any order. Janya ragas (derived from Melakartha ragas) generally have more rigid rules. I used five Melakartha ragas in my machine. If anyone is interested in learning more about how this system works, here are some resources:

– http://www.medieval.org/music/world/carnatic/cmc.html
– https://en.wikipedia.org/wiki/Melakarta

Here is a table of all the 72 Melakartha ragas:

Building this project took a lot more work than I envisioned, but I am extremely happy with the way it turned out. Here is a brief breakdown of how I made the Raga Machine over the last two weeks.

PROTOTYPE STAGES:

My first prototype of the project involved no Arduino aspect at all. It simple involved keys on the keyboard (from ‘z’ to ‘,’) that played one note each.I wanted to take it slow (since we had two weeks to work on it) and see what elements I could get up and working just on Processing.

In the second prototype, I included Arduino buttons. I had eight buttons, and I had each button play a single note that would then change its pitch according to which raga was being played. For both these prototypes, the raga would be changed based on the mouse position.

The visuals for each corresponding raga was something that I had continually struggled with envisioning. For a while, I had each button draw an ellipse of a certain size, with raga determining the color. It looked something like this:

I then changed the visuals to draw various henna/mandala patterns instead of ellipses, but still in random positions with size and color being controlled the same way as before. I was still unclear on what the best course of action was, but luckily, it was time for user testing.

USER TESTING:

Here are the notes I made after my user testing session:

“Even though my project is still missing its decoration component as well as refined visuals, having my friend (who didn’t know what my project was about) come test it out gave rise to some incredibly useful feedback. Here are some of the things I learned and plan to work on:

  1. My project is about Carnatic music, but not many people at this school are aware of what that is, and even if they are, would probably not be able to tell that the project is in fact based off the raga system. My user felt that without context, the project was confusing and vague.
  2. To this end, she suggested making an informational page in the beginning of the project, helping people understand situate the project in some sort of context.
  3. My user also suggested that I label the parts of the musical instrument, but on understanding that I was planning to create a keyboard-like structure for the instrument, she thought that it might need lesser labelling than she originally thought.
  4. She also thought that my visuals were random and confusing, and suggested that the visuals have more to do with the physical input that the user is doing. In line with that, I have changed my visuals to reflect the exact position of the key that the user is pressing at any given moment.”

This feedback turned out to be extremely helpful.

FINAL PRODUCT:

The final product, based on the feedback that I had received both during the user testing session and on the day before the IM show, included vastly different visuals and a pretty, compact box within which the Arduino lay. The keys were made out of popsicle sticks, which then pressed down on the buttons. I thought they would be a drawback but it turned out that a lot of people had fun with them — the idea that they were creating music by pressing down popsicle sticks was entertaining and interesting to them. I also had to add another piece of cardboard atop the structure so that the keys wouldn’t come off the structure.

In addition to these changes, I also added an info page at the beginning, which GREATLY helped during the show to contextualize the entire project. The visuals, too, were labelled with what raga was being played at that moment, which also helped greatly to contextualize the project. Here are two videos, one documenting the info page, and the other documenting a young boy playing the instrument:

Info page:

 

The Instrument:

In hindsight, I think that one of the main components that the project was missing was a signifier. Although most people intuitively understood that the popsicle sticks were meant to function as keys, others would try to pull them out or play them as they would an actual piano. The problem with this was that the popsicle sticks would generate the best sound when pressed where the black dot was, but that wasn’t clear enough of a signifier to make people press that spot immediately. I must say, though, that the visuals helped. Because I changed the ellipses to reflect the position of the key that was being played at the moment, people were much more easily able to grasp what exactly was going on.

All in all, I received extremely good feedback from the visitors at the IM show, and an unexpected number of people were truly interested in learning more about Carnatic music. That, to me, is surely the biggest accomplishment of the project.

As always, major thanks to Nahil and James and Aaron for all the help!

Final Project: User Testing

Even though my project is still missing its hardware component as well as refined visuals, having my friend (who didn’t know what my project was about) come test it out gave rise to some incredibly useful feedback. Here are some of the things I learned and plan to work on:

  1. My project is about Carnatic music, but not many people at this school are aware of what that is, and even if they are, would probably not be able to tell that the project is in fact based off the raga system. My user felt that without context, the project was confusing and vague.
  2. To this end, she suggested making an informational page in the beginning of the project, helping people understand situate the project in some sort of context.
  3. My user also suggested that I label the parts of the musical instrument, but on understanding that I was planning to create a keyboard-like structure for the instrument, she thought that it might need lesser labelling than she originally thought.
  4. She also thought that my visuals were random and confusing, and suggested that the visuals have more to do with the physical input that the user is doing. In line with that, I have changed my visuals to reflect the exact position of the key that the user is pressing at any given moment.

I hope to conduct another round of user testing once my hardware part is done, which would then hopefully result in a more refined and user-friendly project.

 

Response to Golan Levin’s “Computer Vision for Artists and Designers”

Golan Levin’s piece on computer vision was an extremely useful read. The piece is divided into various sections, offering instructions, tips, examples and ideas for beginners dabbling in computer vision. The examples were a startling look into what computer vision was like before the technology of today existed. It was mind blowing, for example, to see the interactive artwork Videoplace and realize that it existed in a time when computer mouses were not a staple. The thought is incredible — looking at Videoplace is like looking at history come alive.

The paper also details certain specific aspects to consider while creating a computer vision project that are helpful guidelines for beginners: detecting motion, detecting presence, detection through brightness thresholding, simple object tracking, and basic interactions. The paper also launches a discussion of computer vision in the physical world — an example of which is the Suicide Box — and discusses how objects and events int he physical world can affect how we parse out the algorithm in our computer vision projects.

I also must mention the large collection of resources that this article presented us with, which I think I will continue to peruse and use in my future ventures with computer vision. Overall, an interesting and super informative read!

Assignment 11: Photobooth

This week, I tried to make a Photobooth sort of program, in which you can see yourself in a variety of colored filters and save an image with the filter you like best. When the program is run, at the top of the screen there are a few colored boxes — hovering the mouse over each of those boxes produces a differently colored filter. On clicking anywhere on the screen, you are able to save the image into a folder on my computer.

These are the different filters in the program:

(Note: the screen is not lined this heavily when it actually runs, I would think these lines are there because of the screen recording.)

Continue reading “Assignment 11: Photobooth”

What Computing Means to Me

When I first decided to take Intro to IM, I did it out of curiosity. It came from a desire to learn about wires and code and how to have fun with those things, while still producing meaningful creations. I also wanted to gain some basic knowledge in computing. Computing, throughout this semester, has challenged me in every way possible. It has challenged my sense of complacency. It has challenged me mentally and emotionally, teaching me to deal with the frustrations of not knowing how to do something. It has challenged my notions of what kind of thinking is valuable, and how I can use this kind of thinking to achieve my goals. So computing, at this point, is a huge teaching moment in my life. As a primarily arts student, learning computing has led me to think in new ways about things that I have learnt in my other classes. I have started to value and cultivate a sort of thinking that is specific and logic-based. But more than anything else, I have started to understand this whole other world of computing, even in my own small way, and the various possibilities that it offers. And that is an eye-opening and wonderful thing.

Assignment 11: What Does Your Painting Sound Like?

Hello, Arduino, I’ve missed you! After having spent a few weeks on Processing now, struggling with code and math and logic, I was so happy to return to a project that required LED lights and buzzers and potentiometers. This week, I decided to make a program that would let users make two art pieces at once: sound art and visual art. Users move the mouse around the screen, making whatever kind of visual elliptic art they want, but at the same time also controlling whichever (pained, tortured) note the buzzer is sounding. Thus, the user can hear their painting, and see their music! Here is how it looks/sounds:

What I like about how the project turned out is the interaction between sound and visual. I love the idea of listening to my painting! I also like the watercolor-like effect created by the borderless ellipses on the Processing screen, giving the illusion of an actual painting.

The hardest part of this project was understanding the logic behind the interaction between Arduino and Processing: I was initially confused about the difference between SerialEvent and other functions, but since my code only needed a one-way interaction (sending mouse position to Arduino from Processing), my code has a function called SendData instead of a SerialEvent. This function is enough to accomplish the interaction that I need, but only because it is super small scale and I am physically here to control both sides.

I thoroughly enjoyed using Arduino again after such a long time, although I had forgotten many of the commands over the weeks. For the future of this particular project, I would want to work further on the details of how the sound is created. I would want it to be more pleasing (pls buzzer), and I would also want the coordination between the mouse and the buzzer to be more intuitive (which it currently is not able to fully be due to the delay function).

Here is the code. As always, thanks to James and Nahil for their patience in answering all my coding questions!

//Arduino:

const int buzzer =3;
int freq = 0;
int duration = 0;

void setup() {
 Serial.begin(9600);
 pinMode(3, OUTPUT);
}

void loop() {
 tone(3, freq, duration);
 delay(200);
}

void serialEvent(){
 while (Serial.available()) {
 freq = Serial.parseInt();
 duration = Serial.parseInt();
 if (Serial.read() == '\n') {
 
 }
 }

}
//Processing:

import processing.serial.*;
Serial myPort;
int rainbowCounter = 0;

void setup () {
 fullScreen();
 background(255);
 printArray(Serial.list());
 String portname=Serial.list()[5];
 println(portname);
 myPort = new Serial(this, portname, 9600);
 myPort.clear();
 myPort.bufferUntil('\n');
}

void draw() {
 int x1 = mouseX;
 int y1 = mouseY;

 noStroke();

 fill(rainbowCounter, 138, 255, random(30, 60));
 if (rainbowCounter == 255) {
 rainbowCounter = 0;
 }
 rainbowCounter++;

 ellipse(x1, y1, random(0, 50), random(0, 40));
 
 sendData();
}


void sendData(){
 int xPos = (int)map(mouseX, 0, width, 400, 900);
 int yPos = (int)map(mouseY, 0, height, 400,900);
 myPort.write(xPos+","+yPos +"\n");
}

Response to “The Digitization of Just About Everything”

In this piece, the author seeks to analyze and understand the digitization of data and media. His chief argument in identifying the what constitutes digitized information is based in two properties: it is non-rival, and it has close to zero marginal cost of reproduction. After this definition of boundaries, the author goes on to discuss various questions that have gained prominence in the wake of digitization of media. He discusses how the information available on the internet is all free of charge, prompting him into a discussion of user-generated output and machine-to-machine interaction. He also discusses the question of data explosion, and reveals some truly mind-boggling math that makes me think about what kind of future this digitization will yield. The author makes sure that the writing is peppered with references, examples and simple language that keeps the reader’s attention focused on the topic at hand. Another clever technique he implements is to make the sections small and concise. As for his concepts in a theoretical sense, I am intrigued by the author’s ideas that classify what constitutes new media, or digitized media. The facts that he has presented about media in the digital age really make me wonder what the future of our media will look like.

Assignment 10: T.Y.P.O.

This week, we created TYPO (Text Your Patterns Out):

Shreya’s Reflection:

This week, our prompt was to either create a data visualization or generate a text output. Zbynek and I, after bouncing ideas off each other, decided to go for text piece that would be both interactive and contain a fun twist. We decided to make a keyboard of sorts, where a person can type in anything they want and the resulting text output will have letters made out of smaller letters.

We set about working on the code. After getting one letter, “A”, to display on the screen as we pressed the key “A”, we moved on to accommodating all the other letters of the alphabet. After that, we started working the patterns of smaller letters that would comprise the patterns in the larger. Initially, they were simple patterns of two or three letters — such as “aa” or “bbb” — and we used these to sort out issues in spacing and width. Finally, we began work on the patterns themselves, first making a grid on paper, and then trying to see how many small letters per row would fit in that grid to make the shape of its corresponding alphabet.

It was a great experience for me to work with Zbynek, who, as a CS major, was able to explain a lot of the logic and math behind the code that we were writing. Understanding the complexity of some of the formulae we used for solving issues of spacing between the letters was probably the biggest challenge for me this week, but I am glad that I have come out of it still able to understand the basic logic underlying it.

Zbynek’s Reflection:

I do not usually enjoy working in groups on programming projects, but it was a great to have Shreya as a sounding board to bounce my ideas off of. Even though I myself did not know how to make the positioning of letters work, she was extremely enthusiastic about getting into the code and really trying to understand the logic. In retrospect, I think we should have gone for an object-oriented approach, encapsulating the math for each element, and working with that, but seeing that we already worked out the positions in the for-loops, the trouble would not have been worth it. Oh well.

I am happy we got to use the function keys in our app, which is something I have not known how to do before. I also enjoyed drawing out the patterns for the keys (although transferring them from paper into the program was a pain). Also, originally, we did not have a way to type a space, but some tweaks to the for-loops took care of that. Now, people can type their names! Like, Shreya, I like the fact that our capital letters are made out of small letters.

If we had more time to do this, we would have supported writing multiple lines of text, and added more patterns – small letters (made out of capitals this time?), numbers, and probably some diacritics (!?.,’). But we are satisfied with the result as it is.

Here is the code:

String [][] alphabetPatterns = {
  {" a "," a a ","a a","aaaaa","a a","a a"}, {"bbbb ","b b","bbbb ","b b","b b","bbbb "}, {" ccc ","c c","c ","c ","c c"," ccc "}, {"ddd ","d d ","d d","d d","d d ","ddd "}, {"eeeee","e ","eee ","e ","e ","eeeee"},
  {"fffff","f ","fff ","f ","f ","f "}, {" ggg ","g g","g ","g ggg","g g"," ggg "}, {"h h","h h","hhhhh","h h","h h","h h"}, {"iiiii"," i "," i "," i "," i ","iiiii"}, {" j"," j"," j"," j","j j"," jjj "},
  {"k k ","k k ","kk ","k k ","k k ","k k"}, {"l ","l ","l ","l ","l ","lllll"}, {"m m","mm mm","m m m","m m","m m","m m"}, {"n n","nn n","n n n","n nn","n n","n n"}, {" ooo ","o o","o o","o o","o o"," ooo "},
  {"pppp ","p p","pppp ","p ","p ","p "}, {" qqq ","q q","q q","q q q","q q "," qq q"}, {"rrrr ","r r","rrrr ","r r ","r r ","r r"}, {" ssss","s "," sss "," s"," s","ssss "}, {"ttttt"," t "," t "," t "," t "," t "},
  {"u u","u u","u u","u u","u u"," uuu "}, {"v v","v v","v v","v v"," v v "," v "}, {"w w","w w","w w","w w w","ww ww","w w"}, {"x x"," x x "," x "," x x ","x x","x x"}, {"y y"," y y "," y "," y "," y "," y "},
  {"zzzzz"," z "," z "," z ","z ","zzzzz"}
};
PFont font;
String s = "";
boolean startScreen = true;

void setup () {
  fullScreen();
  
  textAlign(CENTER);
  font = createFont("courier", 32);
}

void draw () {
  background(0);
  if (startScreen) {
    textFont(font, 32);
    text("Hi! Press ENTER to start typing.", width/2, height/2);
  }
  else {
    float w = width/2 - (((s.length()-1)*20)/2)*alphabetPatterns[0][0].length() - ((s.length()-1)*20)/2 + 10;
    for (int letter = 0; letter < s.length(); letter++) {
      int letterCode = int(s.charAt(letter));
        if (letterCode == 32) {
          w += 20*alphabetPatterns[0][0].length();
          if (letter != s.length()-1) w += 20;
          continue;
        }
        else {
          letterCode -= 97;
          float h = height/2 - ((alphabetPatterns[letterCode].length-1)*30)/2;
          for (int row = 0; row < alphabetPatterns[letterCode].length; row++) {
            float cellW = w - 10 - ((alphabetPatterns[letterCode][row].length()-1)*20)/2;
            for (int cell = 0; cell < alphabetPatterns[letterCode][row].length(); cell++) {
            textFont(font, 32);
            text(alphabetPatterns[letterCode][row].charAt(cell), cellW, h);
            cellW += 20;
          }
          h += 30;
        }
        w += 20*alphabetPatterns[0][0].length();
        if (letter != s.length()-1) w += 20;
      }
    }
  }
}

void keyPressed () {
  if (keyCode == ENTER) startScreen = false;
  
  if (key == ' ') s += " ";
  if (keyCode == DELETE) s = "";
  if (keyCode == BACKSPACE && s.length() >= 1) s = s.substring(0, s.length()-1);

  if (int(key) >= 65 && int(key) <= 90 || int(key) >= 97 && int(key) <= 122) {
    char letter = key;
    if (int(letter) >= 65 && int(letter) <= 90) letter = char(int(letter) + 32);
    s += letter+"";
  }
}

Assignment 9: Balls and Bars

This week, our prompt was to make a game or an art piece using functions and object-oriented programming. I decided to make a simple game involving a bar and a ball — my very first game made with code!

The rules of the game are simple: Use the mouse to move the blue bar and do not let the red ball fall below your bar. If you do, the screen flashes white, indicating that you lost the game. If you want to challenge yourself, you can also hit the spacebar to increase your speed, as many times as you want. Here’s a screen recording of me playing the game:

As you can see, the ball speeds up a bit in the middle because I hit the spacebar a couple of times. The white flashes as soon as I miss the ball, but the game then continues on as before.

Continue reading “Assignment 9: Balls and Bars”

Assignment 8: Recreating Computer Art

This week’s assignment, which was to recreate one of the artworks from this article, was probably the hardest assignment of the semester for me. This, I realized, is due to my inexperience with coding and algorithm-based thinking. Hopefully, after this assignment, I will have a little more experience with those skills that I can put to use in future assignments 🙂

Many artworks in the article caught my eye, but the two I liked best were these:

the Dots piece (whose real name is Untitled Serigraphs):\

the Peacock piece:

However, I tried my hand at both and did not get where I wanted to be — with the first dots one, I could just manage to create a single line of ellipses. With the peacock one, I ended up with a sort of rough draft in which I found I was unable to do the lines within the circles.

My peacock piece draft:

So for Monday’s class, I decided to go for the simple shape-filled artwork. I honestly enjoyed doing it because I could do shapes quite easily, and it was a break from the frustration that I felt while working on the other two.

Here are the shapes:

 

However, I decided that I still really wanted to work on the dots piece, which had been my favorite artwork from the start, and so I got a TON (seriously, a lot) of help and finally arrived at this:

(Sorry for the unclear picture, but the screenshot was even worse.)

It’s not too much like the original, but I’m really quite happy with it because it is so much of an improvement from what I was able to do with it before. The code divides the image into a few sections — the uneven parts at the beginning and end, the two diamonds, and the part between the two. Here is the code:

void setup () {
 size (1000, 640);
 background(0, 0, 0);
 noStroke();
 rect(10, 9, 30, 619);
 rect(10, 598, 980, 30);
 rect(960, 9, 30, 619);
 rect(10, 9, 980, 30);
}

int addition = 0;
void draw() {
 println(mouseX, " ", mouseY);

 int radius =2;
 int spacing =7;
 int counter=544;
 int i = 1;
 int startY = 0;
 int endY = 0;



//first part of screen
 for (int x=63; x<120; x += spacing) {
 //ellipse(x, 329, radius, radius);
 boolean increase = false;
 for (int y=77; y<height-77; y+=spacing) {
 if (y>counter) {
 if (y>77 && y<177) {
 fill(175);
 } else {
 fill(255);
 }
 ellipse(x, y, radius, radius);
 increase = true;
 }
 }
 if (increase == true) {
 if (i == 4)
 counter -= 21;
 else 
 counter -= 7 * (i*7);
 i++;
 }
 }

//middle is 320

int lowD1=200;
int lowD2=0;
int highD1=440;
int highD2=0;

for( int x=120; x<300; x+=spacing){

 lowD1-=spacing/2;
 highD1+=spacing/2;

 if(x%3 ==0){
 //gonna have one that prints to all the way
 
 startY=0;
 endY=height-77;

 }else if(x%3 ==1){
 //one that prints from the top of the diamond to the bottom.
 startY=lowD1;
 endY=height-77;

 }else{

 startY=0;
 endY=highD1;

 }

 for (int y=startY; y<endY; y+=spacing){
 fill(175);
 //figure out shades later.
 ellipse(x, y, radius, radius);
 }


}

lowD2=320;
highD2=320;

for (int x=300; x<500; x+=spacing){

 lowD1-=spacing/2;
 highD1+=spacing/2;
 lowD2-=spacing/2;
 highD2+=spacing/2;


 if(x%3 ==0){
 //this one prints all the way to the bottom from the lowD2; 
 startY=lowD2;
 endY=height-77;

 }else if(x%3 ==1){
 //one that prints the larger diamond shape.
 startY=lowD1;
 endY=highD1;

 }else{
 //one that prints all the way to the top from the high D2;
 startY=0;
 endY=highD2;

 }

 for (int y=startY; y<endY; y+=spacing){
 fill(175);
 ellipse(x, y, radius, radius);
 //figure out shades later.
 }

}



for (int x=500; x<700; x+=spacing){

 lowD1+=spacing/2;
 highD1-=spacing/2;
 lowD2+=spacing/2;
 highD2-=spacing/2;


 if(x%3 ==0){
 //this one prints all the way to the bottom from the lowD2; 
 startY=lowD2;
 endY=height-77;

 }else if(x%3 ==1){
 //one that prints the larger diamond shape.
 startY=lowD1;
 endY=highD1;

 }else{
 //one that prints all the way to the top from the high D2;
 startY=0;
 endY=highD2;

 }

 for (int y=startY; y<endY; y+=spacing){
 fill(175);
 ellipse(x, y, radius, radius);
 //figure out shades later.
 }

}
//second last part of screen

for( int x=700; x<820; x+=spacing){

 lowD1+=spacing/2;
 highD1-=spacing/2;

 if(x%3 ==0){
 //gonna have one that prints to all the way
 
 startY=0;
 endY=highD1;

 }else if(x%3 ==1){
 //one that prints from the top of the diamond to the bottom.
 startY=0;
 endY=height-77;

 }else{
 startY=lowD1;
 endY=height-77;

 }

 for (int y=startY; y<endY; y+=spacing){
 fill(175);
 //add shades.
 ellipse(x, y, radius, radius);
 }


}

//last part of the screen

 counter=544;
 i=1;

for (int x=820; x<950; x+=spacing) {

 boolean increase=true;
 for ( int y=77; y<height-77; y+=spacing) {
 if (y<counter) {
 if (y>77 && y<177) {
 fill(175);
 } else {
 fill(255);
 }
 ellipse(x, y, radius, radius);
 increase=false;
 }
 }
 if (increase==false) {
 if (i == 4)
 counter-=21;
 else 
 counter-=7*(i*7);
 i++;
 }
 }

noLoop();

A few shoutouts:

To Aaron, for being incredibly patient and helping me to set the base for the code.
To UNIX lab friends and lab monitors, for answering all of my silly questions.
To my classmates, who without judgment threw out a bunch of really helpful suggestions to help recreate the dots piece 🙂

Thank you!