Webcam Live Drawing project

I have decided to work on live video for this project, which was mostly inspired by the Computer Vision article as well as I just thought that I can do more things with live video rather than a still image.

 

My initial idea was to track a certain color, say the color of the lips, and then once those pixels were detected, I would change the color of those pixels so that they affect live image. So instead of having pinkish lips, I would be able to change the color to orange, green, or whatever color I would choose in the live video.

Doing this, I faced one problem with the pixel color detection. Because the lips are pink and somewhat similar to the skin tone and the rest of our body, it is very tricky to get just the pixels of the lip color selected.

For example, if I pick just one certain color of the lips and leave the threshold for searching for the similar color in the image as very small, lets say the threshold is 5, then it would select just a tiny bit of the lips:

 

And if I increase the threshold to 25, it would select a lot more than just the lips:

So this made me give up on this idea just because I realized I would not be able to reach the level of accuracy I am looking for. Almost instantly then another idea came to mind, which also involved tracking color, but now I was using tracked color for a different purpose.

 

So the idea is that the program tracks light green color, which in my case is a cap for a pen, and draws a point on every pixel that it finds close to the green color within a threshold of 20.

 

Then, on button press, the program saves the coordinates of those circles, which allows the person who is in the video to draw shapes or whatever (s)he wants while the live video is recording. You can change the colors as well!

This is one the drawings I’ve made, which was fun, but I forgot I had to record a video, so I tried replicating it in the video once again and just having a little fun with it.

 

 

 

I would say that the biggest challenge I faced was figuring out the arrays for colors in a way that when the color is changed, it is changed only for something that is about to be drawn, rather than changing the color of everything that has already been drawn.

 

Here is the code:

import processing.video.*;
Capture video;
color trackColor;
float threshold;
int[] xvalues= {};
int[] yvalues= {};

color currentColorRed;
color currentColorGreen;
color currentColorBlue;
int[] colorsArrayRed= {};
int[] colorsArrayGreen= {};
int[] colorsArrayBlue= {};

boolean saveNow;


void setup() {
 size(640, 480);
 video = new Capture(this, 640, 480, 30);
 video.start();
 trackColor = color(30, 175, 94);
 currentColorRed = 255;
 currentColorGreen = 255;
 currentColorBlue = 255;
}

void draw() {

if (video.available()) {
 video.read();
 }
 video.loadPixels();
 //image(video, 0, 0);

//threshold = map(mouseX, 0, width, 0, 100);
 threshold=20;
 float avgX = 0;
 float avgY = 0;

int count=0;

for (int y=0; y<height; y++) {
 for (int x=0; x<width; x++) {
 int loc = (width-1-x)+(y*width);

//what is the current color
 color currentColor = video.pixels[loc];
 float r1=red(currentColor);
 float g1=green(currentColor);
 float b1=blue(currentColor);
 float r2=red(trackColor);
 float g2=green(trackColor);
 float b2=blue(trackColor);

float d = dist(r1, g1, b1, r2, g2, b2);

if (d<threshold) {
 noStroke();
 strokeWeight(1);
 ellipse(x, y, 10, 10);
 avgX+=x;
 avgY+=y;

if (saveNow ==true) {
 xvalues = append(xvalues, x);
 yvalues = append(yvalues, y);
 colorsArrayRed = append(colorsArrayRed, currentColorRed);
 colorsArrayGreen = append(colorsArrayGreen, currentColorGreen);
 colorsArrayBlue = append(colorsArrayBlue, currentColorBlue);
 }
 count++;
 }
 }
 }
 println(xvalues.length);

if (xvalues.length > 1) {
 for (int i = 0; i < (xvalues.length - 1); i++) {
 fill(colorsArrayRed[i], colorsArrayGreen[i], colorsArrayBlue[i]);
 ellipse(xvalues[i], yvalues[i], 10, 10);
 }
 }
 video.updatePixels();

pushMatrix();
 translate(width, 0);
 scale(-1, 1);
 tint(255, 50);
 image(video, 0, 0);
 tint(255, 255);
 popMatrix();

if (count>0) {
 avgX = avgX/count;
 avgX = avgX/count;
 //fill(trackColor);
 //strokeWeight(4.0);
 //stroke(0);
 //ellipse(avgX, avgY, 8, 8);
 }
}

void keyPressed() {
 if (keyCode == ENTER) {
 saveNow = !saveNow;
 }
 if (key == 'r') {
 currentColorRed = 255; 
 currentColorGreen = 0; 
 currentColorBlue = 0;
 }
 if (key == 'g') { 
 currentColorRed = 0; 
 currentColorGreen = 255; 
 currentColorBlue = 0;
 }
 if (key == 'b') {
 currentColorRed = 0; 
 currentColorGreen = 0; 
 currentColorBlue = 255;
 }
 if (key == 'y') {
 currentColorRed = 255; 
 currentColorGreen = 255; 
 currentColorBlue = 0;
 }
 if (key == 'l') {
 currentColorRed = 0; 
 currentColorGreen = 255; 
 currentColorBlue = 255;
 }
 if (key == 'p') {
 currentColorRed = 255; 
 currentColorGreen = 0; 
 currentColorBlue = 255;
 }
 if (key == 'B') {
 currentColorRed = 0; 
 currentColorGreen = 0; 
 currentColorBlue = 0;
 }
 //if(keyCode == LEFT){
 //int[] colorsArrayRed= {};
 //int[] colorsArrayGreen= {};
 //int[] colorsArrayBlue= {};
 //int[] xvalues= {};
 //int[] yvalues= {};}
 //
}