Final Project: An Image That Can’t Be Vandalized

My final project was born out of two motivations. I wanted to play with the concept of cult of personality, and I wanted to do some sort of projection mapping. I thus decided to make an image that couldn’t be vandalized.

In terms of technical implementation, the project has three main components. The first is an infrared camera (a PS3Eye), which I use to track the position of an infrared LED attached to an object resembling a spray can. The second is the projection: both the equipment used to set it up as well as the things that needed to be done in order to make it work within the spatial constraints. Finally, there is a set of images that are triggered depending on the position of the infrared LED on the canvas–these are perceived by the user as an animation.

IR LED, Camera & Blob Detection

A PImage variable ‘cam’ (640×480) is created to retain whatever is captured by the PS3Eye

A PImage ‘adjustedCam’ (width*height) is created to retain what is being captured in ‘cam’ but in a larger size.

A smaller PImage ‘img’ (80×60) is created to enable the Blob Detection. It is not drawn in the processing sketch but runs in the background. It adjusts the size of ‘adjustedCam’ to effectively restrict what the IR camera can see to the area being projected. This allows a blob to be drawn in the same place as where the IR LED is turned on.

Setting the coordinates.

A circuit connected to an IR LED is built into a Pringles can adapted to resemble a spray can. I used a weight to resemble the sensation of holding a spray can, and a ping pong ball to mimic the sound.

Spray can circuit and design.

I use Blob Detection — a form of pixel manipulation that sorts bright from non-bright pixels — to track the position of the IR LED over the canvas. The presence of a Blob–which indicated that a light is ON–triggers a drawing over the position of the light.

Projection Setup
The most time-consuming aspect of the project. Setting up in the space and adjusting the projector’s elevation over the ground and its distance from the wooden canvas. I used to film-set stands to hold the wooden frame.

Projection setup in the IM lab, with the wooden frame.

Animation
There are two components of the animation: what happens when the user ‘sprays’ on inside the painting and when they don’t.
When they are spraying outside the painting, the painting’s character follows the position of the spray can with his eyes. This I do by mapping the position of two ellipses drawn in the eyes of the character to the position of the blob.

When spraying happens inside the portrait, different frames get triggered depending on the general position of the blob.

Notes from user testing
My user-testing pointed me toward the following things which I implemented in the final project.

  • Add weight to the spray can and protect the circuit because people will want to shake the can — allow them to have that experience.
  • Allow the users to change the color of the spray paint.
  • Make the character in the painting dock.

IM Showcase

Here are some pictures of the IM showcase and the accumulated paintings that resulted form people interacting with my piece.


// - Super Fast Blur v1.1 by Mario Klingemann <http://incubator.quasimondo.com>
// - BlobDetection library

import processing.video.*;
import blobDetection.*;

BlobDetection theBlobDetection;
PImage img;
boolean newFrame=false;
import com.thomasdiewald.ps3eye.PS3EyeP5;

PS3EyeP5 ps3eye;
PImage cam;

PImage adjustedCam; // adjustedCam image

float posX; // for tracking the position of the blob
float posY;

float eyesXleft, eyesYleft; // positioning pupils
float eyesXright, eyesYright;

// images
PImage frame1; // the frames for animation
PImage frame2;
PImage frame3;
PImage frame4;
PImage frame5;
PImage frame6;
PImage frame7;
PImage pic_frame;
PImage noEyes; // image for static/eye-drawing
PImage rainbow; // rainbow color option square

int colPaint;
//int rainbowPaint;
boolean rainbowPaint;

boolean mode; // a boolean indicating whether
// the blob is moving inside or
// outside the frame

void setup()
{
fullScreen(P3D); // a 640*480 resolution of the screen matches the IR camera
//size(1280,720,P3D);
//fullScreen();
//size(640,480);
ps3eye = PS3EyeP5.getDevice(this);

if (ps3eye == null) {
//System.out.println(“No PS3Eye connected. Good Bye!”);
exit();
return;
}

// start capturing with 60 fps (default)
ps3eye.start();

// BlobDetection
// img which will be sent to detection (a smaller copy of the cam frame);
cam=createImage(640, 480, RGB);
img = new PImage(80, 60);
adjustedCam = createImage(width, height, RGB);
theBlobDetection = new BlobDetection(img.width, img.height);
theBlobDetection.setPosDiscrimination(true);
theBlobDetection.setBlobMaxNumber(1);
theBlobDetection.setThreshold(0.05f); // will detect bright areas whose luminosity > 0.2f;

//===loading images
frame1 = loadImage(“frame1.png”);
frame2 = loadImage(“frame2.png”);
frame3 = loadImage(“frame3.png”);
frame4 = loadImage(“frame4.png”);
frame5 = loadImage(“frame5.png”);
frame6 = loadImage(“frame6.png”);
frame7 = loadImage(“frame7.png”);
pic_frame = loadImage(“frame_picture.png”);
rainbow = loadImage(“rainbow.png”);
noEyes = loadImage(“noEyes.png”);

//===setting intial color
colPaint = color(255,0,0);
rainbowPaint = false;
}

void draw()
{

if (ps3eye.isAvailable()) {
cam = ps3eye.getFrame();
}

adjustedCam.copy(cam, 0, 0, cam.width, cam.height, 0, 0, adjustedCam.width, adjustedCam.height);
int beginX=247; // for IM show
int beginY=146;
int endX=1037;
int endY=498;

//int beginX=272; // for IM show
//int beginY=169;
//int endX=1099;
//int endY=503;

//int beginX=289;
//int beginY=168;
//int endX=1116;
//int endY=510;

//int beginX=198;
//int beginY=131;
//int endX=396;
//int endY=266;

img.copy(adjustedCam, beginX, beginY, endX-beginX, endY-beginY, 0, 0, img.width, img.height);
//img.copy(adjustedCam, 0, 0, adjustedCam.width, adjustedCam.height, 0, 0, img.width, img.height);
//img.copy(cam, 225, 140, 390-225, 261-140, 0, 0, img.width, img.height);
fastblur(img, 2);
//image(cam, 0,0, width,height);
//fastblur(cam,2);

//float threshold =50;

//img.loadPixels();
//adjustedCam.loadPixels();

//for (int x = 0; x < img.width; x++) {
// for (int y = 0; y < img.height; y++ ) { // int loc = x + y*img.width; // // Test the brightness against the threshold // if (brightness(img.pixels[loc]) > threshold) {
// adjustedCam.pixels[loc] = color(255); // White
// } else {
// adjustedCam.pixels[loc] = color(0); // Black
// }
// }
//}
//img.updatePixels();
//adjustedCam.updatePixels();
theBlobDetection.computeBlobs(img.pixels);
//image(img,0,0, width, height); // comment
drawBlobsAndEdges(false, false, true);
// Display the adjustedCam

image(pic_frame, 0, 0, width, height);
image(frame1, 298, 147, 685, 426);

// detecting if there is a blob or not; to trigger animations
if(theBlobDetection.getBlobNb()>=1){

eyesXleft = map(posX,0,width,604,624);
eyesYleft = map(posY,0,height,325,332);

eyesXright = map(posX,0,width,650,676);
eyesYright = map(posY,0,height,325,336);

// determining MODE. TRUE = animation, FALSE = eye tracking
if (posX>=0 && posX<=367 && posY>=0 && posY<=height || // left area posX>=912 && posX<=width && posY>=0 && posY<=height || // right area posX>=368 && posX<=911 && posY>=0 && posY<=227 || // upper area posX>=368 && posX<=911 && posY>=575 && posY<=height // lower area
) {
mode = false;
} else {
mode = true;
}
if (mode == false) {
image(noEyes, 298, 147, 685, 426);
noStroke();
fill(0);
ellipse(eyesXleft, eyesYleft, 5, 5);
fill(0);
ellipse(eyesXright, eyesYright, 5, 5);

noStroke();
if (rainbowPaint==false){ // to set the color either as rainbow or as solid fill
fill(colPaint);
} else {
rainbowPaint=true;
fill(random(0,255),random(0,255),random(0,255));
}
for(int i = 0; i < 5; i++){ // this gives the graffiti-looking effect
float randX = random(0,20);
randX = randX – 10;

float randY = random(0,20);
randY = randY – 10;

ellipse(posX+randX, posY + randY, 3,3);
}
}
if (mode == true) { //changing the animation frames
if (457 <= posX && posX <= 548 && 303 <= posY && posY <= 515) {
image(frame2, 298, 147, 685, 426);
} else if (638 <= posX && posX <= 723 && 303 <= posY && posY <= 515) {
image(frame5, 298, 147, 685, 426);
} else if (725 <= posX && posX <= 815 && 303 <= posY && posY <= 515) {
image(frame4, 298, 147, 685, 426);
} else if (549 <= posX && posX <= 636 && 303 <= posY && posY <= 515) {
image(frame3, 298, 147, 685, 426);
} else if (458 <= posX && posX <= 816 && 207 <= posY && posY <= 254) {
image(frame6, 298, 147, 685, 426);
} else if (458 <= posX && posX <= 816 && 255 <= posY && posY <= 303) {
image(frame7, 298, 147, 685, 426);
}else {
image(frame1, 298, 147, 685, 426);
}
noStroke();
noFill();
ellipse(posX, posY, 10, 10);
}
} else {
}

////===Color palette

fill(random(0,255),random(0,255),random(0,255));

noStroke();

fill(255,0,0); // color 1
rect(0+15, height-50, 40, 40);

fill(0); // color 2
rect(0+65, height-50, 40, 40);

fill(0,0,255); // color 3
rect(0+115, height-50, 40, 40);

//rect(0+165, height-50, 40, 40); // rainbow
image(rainbow, 0+165, height-50, 40, 40); // rainbow

if (16 <= posX && posX <= 55 && 672 <= posY && posY <= 712){
rainbowPaint = false;
colPaint = color(255,0,0);
} else if (66 <= posX && posX <= 105 && 672 <= posY && posY <= 712){
rainbowPaint = false;
colPaint = color(0);
} else if (106 <= posX && posX <= 155 && 672 <= posY && posY <= 712){
rainbowPaint = false;
colPaint = color(0,0,255);
}else if (166 <= posX && posX <= 205 && 672 <= posY && posY <= 712){
rainbowPaint = true;
} else {
}

//fill(0,255,0,100); // for checking projection map
//rect(0,0,width,height);

}
//
// ==================================================
// get the coordinates of the projection — for mapping
// ==================================================

void mousePressed() {
println(mouseX, mouseY);
// prints the coordinates of where the mouse is
// pressed; the coords of the projection.
}

// ==================================================
// drawBlobsAndEdges()
// ==================================================
void drawBlobsAndEdges(boolean drawBlobs, boolean drawEdges, boolean getCoordinates)
{
noFill();
Blob b;
EdgeVertex eA, eB;
for (int n=0; n<theBlobDetection.getBlobNb(); n++) {
b=theBlobDetection.getBlob(n);
if (b!=null) {
//Edges

if (drawEdges) {
strokeWeight(3);
stroke(0, 255, 0);

for (int m=0; m<b.getEdgeNb(); m++) {
eA = b.getEdgeVertexA(m);
eB = b.getEdgeVertexB(m);

if (eA !=null && eB !=null) {

line(
eA.x*width, eA.y*height,
eB.x*width, eB.y*height
);
}
}
}

// Blobs
if (drawBlobs) {

fill(255, 150);
ellipse(b.x*width, b.y*height, 30, 30);

strokeWeight(1);
stroke(255, 0, 0);
rect(
b.xMin*width, b.yMin*height,
b.w*width, b.h*height
);
}

posX = b.x*width;
posY = b.y*height;
//println(“posX”);
//println(posX);
//println(“posY”);
//println(posY);
}
}
}

// ==================================================
// Super Fast Blur v1.1
// by Mario Klingemann
// <http://incubator.quasimondo.com>
// ==================================================
void fastblur(PImage img, int radius)
{
if (radius<1) {
return;
}
int w=img.width;
int h=img.height;
int wm=w-1;
int hm=h-1;
int wh=w*h;
int div=radius+radius+1;
int r[]=new int[wh];
int g[]=new int[wh];
int b[]=new int[wh];
int rsum, gsum, bsum, x, y, i, p, p1, p2, yp, yi, yw;
int vmin[] = new int[max(w, h)];
int vmax[] = new int[max(w, h)];
int[] pix=img.pixels;
int dv[]=new int[256*div];
for (i=0; i<256*div; i++) {
dv[i]=(i/div);
}

yw=yi=0;

for (y=0; y<h; y++) {
rsum=gsum=bsum=0;
for (i=-radius; i<=radius; i++) { p=pix[yi+min(wm, max(i, 0))]; rsum+=(p & 0xff0000)>>16;
gsum+=(p & 0x00ff00)>>8;
bsum+= p & 0x0000ff;
}
for (x=0; x<w; x++) { r[yi]=dv[rsum]; g[yi]=dv[gsum]; b[yi]=dv[bsum]; if (y==0) { vmin[x]=min(x+radius+1, wm); vmax[x]=max(x-radius, 0); } p1=pix[yw+vmin[x]]; p2=pix[yw+vmax[x]]; rsum+=((p1 & 0xff0000)-(p2 & 0xff0000))>>16;
gsum+=((p1 & 0x00ff00)-(p2 & 0x00ff00))>>8;
bsum+= (p1 & 0x0000ff)-(p2 & 0x0000ff);
yi++;
}
yw+=w;
}

for (x=0; x<w; x++) {
rsum=gsum=bsum=0;
yp=-radius*w;
for (i=-radius; i<=radius; i++) {
yi=max(0, yp)+x;
rsum+=r[yi];
gsum+=g[yi];
bsum+=b[yi];
yp+=w;
}
yi=x;
for (y=0; y<h; y++) {
pix[yi]=0xff000000 | (dv[rsum]<<16) | (dv[gsum]<<8) | dv[bsum];
if (x==0) {
vmin[y]=min(y+radius+1, hm)*w;
vmax[y]=max(y-radius, 0)*w;
}
p1=x+vmin[y];
p2=x+vmax[y];

rsum+=r[p1]-r[p2];
gsum+=g[p1]-g[p2];
bsum+=b[p1]-b[p2];

yi+=w;
}
}
}