Assignment 13: Zen

Interactions with the final version of my final project are presented below:

Zen is a zen-garden simulation that invites users to slow down and relax during the hectic time of final exams and projects. It does so by presenting an outline of the user in a field of flowers (thank you Lama for making such beautiful plants!), and allowing one to wade through. If one moves too much, the color changes from green to yellow to red, which has consequences on the growth of the plants in the garden.

In the red state, the plants do not grow, and touching them with one’s hands or feet causes them to wither. It is only when the user slows down and reaches the green state that they get to experience the reward – planting their own flowers for everyone to see!

If one stays calm for a short while, pink lotus plants spawn in the regions occupied by what the Kinect detector sees as people. These can be picked up by people’s hands, and planted in the garden.

If one stays calm for a slightly longer while, a purple plant sprouts in one of the person’s hands; the person can plant those as well! As can be seen from the video, these plants are easier to plant (because the person does not need to actively pick up the plant with their hand).

I am very happy that I was able to implement most of the changes requested during user testing! Planting was a big part of the challenge, and I had to rework the majority of the code to make it possible; after making it happen for the purple plants however, adding the lotus plants was very easy.

Additional signifiers were added to tell people what they should try doing with their plants – this helped explain the interaction better, but I am afraid people still did not have enough patience to wait and see what happens. Also, markings were added to the floor that showed people precisely where they should stand in order to be seen by the Kinect camera (blue area), and precisely where they should plant their flowers (green area with a plant symbol). This proved fairly intuitive.

Unfortunately, I did not have enough time to implement the wind-like effects that would bend the flowers with people’s movement. To make the interaction more intuitive, I added the functionality to shrink flowers when they come in contact with a user’s hand. This is not precisely an intuitively expected interaction, but it did engage the users and showed them that there is something that can be done with the visualization, that it is not just a static arrangement of flowers.

I was struggling with interference from people behind my detection area. The other visualization was too close and the Kinect was mistakenly detecting those users as my users. This was a problem because the visualization beyond my detection area had a very long interaction – so people, once detected, would not be un-detected unless the Kinect was forced to forget them (by me blocking their body with my body). The problem was alleviated a little by tweaks to the code that ensured that only one person would be tracked by the Kinect at a time (which was difficult because the library code I was using did not work as expected), but the visualization still required constant surveillance on my part – which is obviously not ideal. I realized too late that I should have requested a blanket or a screen to prevent background people from interfering with the visualization…

Nevertheless, I am pleased with the end result, I think people liked it much more than they did during user testing, and I think they appreciated the ability to leave their mark for others to see in the visualization.

The code is presented below. It is the longest I have written for this class, overcoming even the CM Visualizations project.

import kinect4WinSDK.Kinect;
import kinect4WinSDK.SkeletonData;

Kinect kinect;
ArrayList<SkeletonData> bodies;

PImage kinectDepth;
PImage kinectMask;

int backgroundR = 12;
int backgroundG = 27;
int backgroundB = 16;
color backgroundColor = color(backgroundR, backgroundG, backgroundB);

int numPixels;
int[] previousFrame;
int movementSum = 0;

int MAX_GROUND_BRANCHES = 100;
ArrayList<Branch> groundBranches;

ArrayList<Branch> leftHandBranches;
ArrayList<Branch> rightHandBranches;

ArrayList<Branch> plantedBranches;

ArrayList<Branch> bodyBranches;

boolean leftHandPlantingBodyBranch = false;
boolean rightHandPlantingBodyBranch = false; 

Float[] dummyLine = {width*2.0, height*2.0, width*2.0, height*2.0};

ArrayList<Float[][]> skeletonLines;
Float[][] dummySkeletonLines = {dummyLine, dummyLine, dummyLine, dummyLine, dummyLine};

int state = 0;
int stateLength = 0;
int desiredState = 0;
int desiredStateLength = 0;
int DESIRED_STATE_MIN_LENGTH = 5;

PFont messageFont;
PFont infoFont;

void setup() {
  fullScreen();
  background(backgroundColor);
  kinect = new Kinect(this);
  smooth();
  bodies = new ArrayList<SkeletonData>();
  
  numPixels = width * height;
  previousFrame = new int[numPixels];
  
  groundBranches = new ArrayList<Branch>();
  
  leftHandBranches = new ArrayList<Branch>();
  rightHandBranches = new ArrayList<Branch>();
  
  plantedBranches = new ArrayList<Branch>();
  
  bodyBranches = new ArrayList<Branch>();
  
  makeGroundBranches();
  
  skeletonLines = new ArrayList<Float[][]>();
  
  messageFont = createFont("GothamUltra Regular.otf", 128);
  infoFont = createFont("GothamUltra Regular.otf", 24);
}

void draw() { 
  kinectDepth = kinect.GetDepth();
  kinectMask = kinect.GetMask();
  
  kinectDepth.resize(width,height);
  kinectMask.resize(width,height);
  
  background(backgroundColor);
  
  kinectDepth.loadPixels();
  kinectMask.loadPixels();
  
  // preparation
  getMovementSum(); 
  getState();
  
  updateSkeletonLines();
  
  checkCollisionsGroundBranches(); 
  checkFrontGroundBranches();
  
  adjustGroundBranches(); // avoiding hands and body
  adjustHandBranches(); // move with hands
  adjustPlantedBranches();
  adjustBodyBranches(); // spawn if not person not moving
 
  drawBehindGroundBranches();
  drawPersonFrameDiff();
  drawPersonBones();
  drawBodyBranches();
  drawHandBranches();
  drawFrontGroundBranches();
  drawMessage();
  drawInfo();
}

void getMovementSum() {
  movementSum = 0;
  for (int y = 0; y < height; y += 1) {
    for (int x = 0; x < width; x += 1) {
      int loc = x + y*width;
      
      boolean isMask = (alpha(kinectMask.pixels[loc]) != 0);
      
      if (isMask) {
        int depth = int(brightness(kinectDepth.pixels[loc]));
        
        int diff = abs(depth - previousFrame[loc]);
        movementSum += diff;
      }
    }
  }
}

void getState() {
  if (movementSum == 0) {
    state = 0;
    if (desiredState != 0) {
      desiredState = 0;
      desiredStateLength = 0;
    }
    else {
      desiredStateLength += 1;
    }
  }
  
  else if (movementSum < 5000000) {
    if (state == 0) {
      state = 1;
    }
    if (desiredState != 1) {
      desiredState = 1;
      desiredStateLength = 0;
    }
    else {
      desiredStateLength += 1;
    }
  }
  
  else if (movementSum < 10000000) {
    if (state == 0) {
      state = 2;
    }
    if (desiredState != 2) {
      desiredState = 2;
      desiredStateLength = 0;
    }
    else {
      desiredStateLength += 1;
    }
  }
  
  else {
    if (state == 0) {
      state = 3;
    }
    if (desiredState != 3) {
      desiredState = 3;
      desiredStateLength = 0;
    }
    else {
      desiredStateLength += 1;
    }
  }
  
  if (desiredState == state) {
    stateLength += 1;
  }
  else {
    if (desiredStateLength >= DESIRED_STATE_MIN_LENGTH) {
      state = desiredState;
      stateLength = 0;
    }
    else {
      stateLength += 1;
    }
  }
}

void updateSkeletonLines() {
  synchronized (bodies) {
    synchronized (skeletonLines) { 
      // get current skeleton lines
      for (int i = 0; i < bodies.size(); i += 1) {
        skeletonLines.set(i, getSkeletonLines(bodies.get(i)));
      }
    }
  }
}

void checkCollisionsGroundBranches() {
  // collide branches with person
  for (int i = 0; i < groundBranches.size(); i += 1) {
    Branch currentBranch = groundBranches.get(i);
    
    currentBranch.isCollision = false; // reset isCollision state
    for (int j = 0; j < bodies.size(); j+= 1) {
      Float[][] currentSkeletonLines = skeletonLines.get(j);
      // check for intersection with feet
      currentBranch.checkIsCollision(currentSkeletonLines[3], currentSkeletonLines[4]);
      currentBranch.checkIsCollision(currentSkeletonLines[1], currentSkeletonLines[2]);
    }
  }
  
  for (int i = 0; i < plantedBranches.size(); i += 1) {
    Branch currentBranch = plantedBranches.get(i);
    
    currentBranch.isCollision = false; // reset isCollision state
    for (int j = 0; j < bodies.size(); j+= 1) {
      Float[][] currentSkeletonLines = skeletonLines.get(j);
      // check for intersection with feet
      currentBranch.checkIsCollision(currentSkeletonLines[3], currentSkeletonLines[4]);
      currentBranch.checkIsCollision(currentSkeletonLines[1], currentSkeletonLines[2]);
    }
  }
}

void checkFrontGroundBranches() {
  // is branch behind or before person?
  for (int i = 0; i < groundBranches.size(); i += 1) {
    Branch currentBranch = groundBranches.get(i);
    
    currentBranch.isInFront = false; // resetIsInFront state
    for (int j = 0; j < bodies.size(); j += 1) {
      Float[][] currentSkeletonLines = skeletonLines.get(j);
      
      currentBranch.checkIsInFront(currentSkeletonLines[3], currentSkeletonLines[4]);
    }
  }
  
  for (int i = 0; i < plantedBranches.size(); i += 1) {
    Branch currentBranch = plantedBranches.get(i);
    
    currentBranch.isInFront = false; // resetIsInFront state
    for (int j = 0; j < bodies.size(); j += 1) {
      Float[][] currentSkeletonLines = skeletonLines.get(j);
      
      currentBranch.checkIsInFront(currentSkeletonLines[3], currentSkeletonLines[4]);
    }
  }
}

void adjustGroundBranches() {
  adjustGroundBranchSizes();
}

void adjustGroundBranchSizes() {
  for (int i = 0; i < groundBranches.size(); i += 1) {
    Branch currentBranch = groundBranches.get(i);
    float scale = currentBranch.scale;
    boolean isCollision = currentBranch.isCollision;
    if (state == 0) {
      scale += 0.025;
      if (scale > 1.0) scale = 1.0;
    }
    else if (state == 1) {
      scale += 0.025;
      if (scale > 1.0) scale = 1.0;
    }
    else if (state == 2) {
      if (isCollision) {
        scale -= 0.01;
        if (scale < 0.0) scale = 0.0;
      }
    }
    else {
      if (isCollision) {
        scale -= 0.1;
        if (scale < 0.0) scale = 0.0;
      }
    }
    currentBranch.scale = scale;
  }
}

void adjustHandBranches() {
  // spawn new
  float leafFatness = random(1, 2);
  int[] leafColorBounds = {127, 196, 0, 0, 0, 255};
  if (state == 1 && stateLength >= 100 && stateLength%100 == 0) {
    if (random(2) >= 1) {
      if (leftHandBranches.size() == 0) leftHandBranches.add(new Branch(random(1), 0.3, leafFatness, leafColorBounds));
    }
    else {
      if (rightHandBranches.size() == 0) rightHandBranches.add(new Branch(random(1), 0.3, leafFatness, leafColorBounds));
    }
  }
  
  // adjust position and plant
  boolean leftHandBranchRemoved = false;
  boolean rightHandBranchRemoved = false;
  for (int i = skeletonLines.size()-1; i >= 0; i -= 1) {
    Float[][] currentSkeletonLines = skeletonLines.get(i);
    
    for (int j = 0; j < leftHandBranches.size(); j += 1) {
      Branch currentBranch = leftHandBranches.get(j);
      currentBranch.adjustPosition(currentSkeletonLines[1]);
      
      if (leftHandBranchRemoved) continue;
      if (state != 0) {
        if (currentBranch.modelLine[0] >= 20 && currentBranch.modelLine[0] < width-20 && currentBranch.modelLine[1] >= height-height/5 && currentBranch.modelLine[1] < height) {
          if (leftHandPlantingBodyBranch) plantedBranches.add(new Branch(new PVector(currentBranch.modelLine[0], currentBranch.modelLine[1]), currentBranch.leafFatness, currentBranch.leafColorBounds));
          else plantedBranches.add(new Branch(new PVector(currentBranch.modelLine[0], currentBranch.modelLine[1]), 0.6, currentBranch.leafFatness, currentBranch.leafColorBounds));
          leftHandBranches.remove(j);
          leftHandBranchRemoved = true;
          leftHandPlantingBodyBranch = false;
        }
      }
    }
    
    for (int j = 0; j < rightHandBranches.size(); j += 1) {
      Branch currentBranch = rightHandBranches.get(j);
      currentBranch.adjustPosition(currentSkeletonLines[2]);
      
      if (rightHandBranchRemoved) continue;
      if (state != 0) {
        if (currentBranch.modelLine[0] >= 20 && currentBranch.modelLine[0] < width-20 && currentBranch.modelLine[1] >= height-height/5 && currentBranch.modelLine[1] < height) {
          if (rightHandPlantingBodyBranch) plantedBranches.add(new Branch(new PVector(currentBranch.modelLine[0], currentBranch.modelLine[1]), currentBranch.leafFatness, currentBranch.leafColorBounds));
          else plantedBranches.add(new Branch(new PVector(currentBranch.modelLine[0], currentBranch.modelLine[1]), 0.6, currentBranch.leafFatness, currentBranch.leafColorBounds));
          rightHandBranches.remove(j);
          rightHandBranchRemoved = true;
          rightHandPlantingBodyBranch = false;
        }
      }
    }
  }
  
  // adjust size
  adjustHandBranchSizes();
}

void adjustHandBranchSizes() {
  for (int i = 0; i < leftHandBranches.size(); i += 1) {
    Branch currentBranch = leftHandBranches.get(i);
    float scale = currentBranch.scale;
    if (state == 0) {
      scale -= 1.0;
      if (scale < 0.0) scale = 0.0;
    }
    else if (state == 1) {
      scale += 0.1;
      if (scale > 1.0) scale = 1.0;
    }
    else if (state == 2) {
      scale -= 0.0;
      if (scale < 0.0) scale = 0.0;
    }
    else {
      scale -= 0.001;
      if (scale < 0.0) scale = 0.0;
    }
    currentBranch.scale = scale;
  }
  
  for (int i = 0; i < rightHandBranches.size(); i += 1) {
    Branch currentBranch = rightHandBranches.get(i);
    float scale = currentBranch.scale;
    if (state == 0) {
      scale -= 1.0;
      if (scale > 1.0) scale = 1.0;
    }
    else if (state == 1) {
      scale += 0.1;
      if (scale > 1.0) scale = 1.0;
    }
    else if (state == 2) {
      scale -= 0.0;
      if (scale < 0.0) scale = 0.0;
    }
    else {
      scale -= 0.001;
      if (scale < 0.0) scale = 0.0;
    }
    currentBranch.scale = scale;
  }
}

void adjustPlantedBranches() {
  adjustPlantedBranchSizes();
}

void adjustPlantedBranchSizes() {
  for (int i = 0; i < plantedBranches.size(); i += 1) {
    Branch currentBranch = plantedBranches.get(i);
    float scale = currentBranch.scale;
    boolean isCollision = currentBranch.isCollision;
    if (state == 0) {
      scale += 0.005;
      if (scale > 1.0) scale = 1.0;
    }
    else if (state == 1) {
      scale += 0.01;
      if (scale > 1.0) scale = 1.0;
    }
    else if (state == 2) {
      if (isCollision) {
        scale -= 0.025;
        if (scale < 0.0) scale = 0.0;
      }
    }
    else {
      if (isCollision) {
        scale -= 0.25;
        if (scale < 0.0) scale = 0.0;
      }
      
      scale -= 0.001;
      if (scale < 0.0) scale = 0.0;
    }
    currentBranch.scale = scale;
  }
}

void adjustBodyBranches() {
  // spawn new
  float leafFatness = random(1, 2);
  int[] leafColorBounds = {255, 255, 0, 191, 63, 127};
  if (state == 1 && stateLength >= 20 && stateLength%10 == 0) {
    bodyBranches.add(new Branch(findPosOnBody(), leafFatness, leafColorBounds));
  }
  
  // adjust size
  adjustBodyBranchSizes();
  
  // attach to hands
  boolean bodyBranchAttached = false;
  for (int i = skeletonLines.size()-1; i >= 0; i -= 1) {
    Float[][] currentSkeletonLines = skeletonLines.get(i);
    
    for (int j = 0; j < bodyBranches.size(); j += 1) {
      Branch currentBranch = bodyBranches.get(j);
      
      if (bodyBranchAttached) continue;
      if (state != 0) {
        if (leftHandBranches.size() == 0 && currentBranch.checkModelLineIntersection(currentSkeletonLines[1])) {
          leftHandBranches.add(new Branch(random(1), currentBranch.leafFatness, currentBranch.leafColorBounds));
          bodyBranches.remove(j);
          bodyBranchAttached = true;
          leftHandPlantingBodyBranch = true;
        }
        else if (rightHandBranches.size() == 0 && currentBranch.checkModelLineIntersection(currentSkeletonLines[2])) {
          rightHandBranches.add(new Branch(random(1), currentBranch.leafFatness, currentBranch.leafColorBounds));
          bodyBranches.remove(j);
          bodyBranchAttached = true;
          rightHandPlantingBodyBranch = true;
        }
      }
    }
  }
}

void adjustBodyBranchSizes() {
  if (state != 1) {
    for (int i = 0; i < bodyBranches.size(); i += 1) {
      Branch currentBranch = bodyBranches.get(i);
      currentBranch.scale -= 0.25;
      if (currentBranch.scale < 0.0) bodyBranches.remove(i);
    }
  }
  else {
    for (int i = 0; i < bodyBranches.size(); i += 1) {
      Branch currentBranch = bodyBranches.get(i);
      if (checkIsPosOnBody(currentBranch.pos)) { // if branch is still on body, grow it
        currentBranch.scale += 0.05;
        if (currentBranch.scale > 1.0) currentBranch.scale = 1.0;
      }
      else { // otherwise, make it wither
        currentBranch.scale -= 0.25;
        if (currentBranch.scale < 0.0) bodyBranches.remove(i);
      }
    }
  }
}

PVector findPosOnBody() {
  PVector pos = new PVector(width, height);
  boolean isMask = false;
  while (isMask == false) {
    pos.x = int(random(width));
    pos.y = int(random(height));
    isMask = checkIsPosOnBody(pos);
  }
  return pos;
}

boolean checkIsPosOnBody(PVector pos) {
  int loc = int(pos.x) + int(pos.y*width);
  boolean isMask = (alpha(kinectMask.pixels[loc]) != 0);
  return isMask;
}

void drawBehindGroundBranches() {
  // draw branches that are behind person
  for (int i = 0; i < groundBranches.size(); i += 1) {
    Branch currentBranch = groundBranches.get(i);
    if (!currentBranch.isInFront) currentBranch.display();
  }
  
  for (int i = 0; i < plantedBranches.size(); i += 1) {
    Branch currentBranch = plantedBranches.get(i);
    if (!currentBranch.isInFront) currentBranch.display();
  }
}

void drawPersonFrameDiff() {
  loadPixels();
  if (state != 0) {
  for (int y = 0; y < height; y += 1) {
    for (int x = 0; x < width; x += 1) {
      int loc = x + y*width;
      
      boolean isMask = (alpha(kinectMask.pixels[loc]) != 0);
      
      if (isMask) {
        int depth = int(brightness(kinectDepth.pixels[loc])); 
 
        int diff = abs(depth - previousFrame[loc]);
        if (diff != 0) {
          if (state == 1) pixels[loc] = color(backgroundR, backgroundG+(diff-backgroundG)*0.5, backgroundB);
          if (state == 2) pixels[loc] = color(backgroundR+(diff-backgroundR)*0.5, backgroundG+(diff-backgroundG)*0.5, backgroundB);
          if (state == 3) pixels[loc] = color(backgroundR+(diff-backgroundR)*0.5, backgroundG, backgroundB);
        }
        else {
          pixels[loc] = backgroundColor;
        }
        
        previousFrame[loc] = depth;
      }
      else {
        previousFrame[loc] = 0;
      }
    }
  }
  updatePixels();
  }
}

void drawPersonBones() {
  if (state != 0) {
    for (int i = 0; i < skeletonLines.size(); i += 1) {
      Float[][] currentSkeletonLines = skeletonLines.get(i);
      drawBones(currentSkeletonLines);
    }
  }
}

void drawBodyBranches() {
  for (int i = 0; i < bodyBranches.size(); i += 1) {
    Branch currentBranch = bodyBranches.get(i);
    currentBranch.display();
  }
}

void drawHandBranches() {
  if (state != 0) {
    for (int i = 0; i < skeletonLines.size(); i += 1) {
      Float[][] currentSkeletonLines = skeletonLines.get(i);
      
      for (int j = 0; j < leftHandBranches.size(); j += 1) {
        Branch currentBranch = leftHandBranches.get(j);
        currentBranch.adjustPosition(currentSkeletonLines[1]);
        currentBranch.display();
      }
      
      for (int j = 0; j < rightHandBranches.size(); j += 1) {
        Branch currentBranch = rightHandBranches.get(j);
        currentBranch.adjustPosition(currentSkeletonLines[2]);
        currentBranch.display();
      }
    }
  }
}

void drawFrontGroundBranches() {
  // draw branches that are in front of person
  for (int i = 0; i < groundBranches.size(); i += 1) {
    Branch currentBranch = groundBranches.get(i);
    if (currentBranch.isInFront) currentBranch.display();
  }
  
  for (int i = 0; i < plantedBranches.size(); i += 1) {
    Branch currentBranch = plantedBranches.get(i);
    if (currentBranch.isInFront) currentBranch.display();
  }
}

void drawMessage() {
  String s = "";
  textFont(messageFont);
  textAlign(CENTER,CENTER);
  noStroke();
  if (state == 0) { // no person
    fill(255,255,255);
    s = "STEP INTO THE JUNGLE";
  }
  else if (state == 1) { // little movement
    fill(0,255,0);
    s = "BREATHE";
  }
  else if (state == 2) { // medium movement
    fill(255,255,0);
    s = "SLOW DOWN";
  }
  else { // high movement
    fill(255,0,0);
    s = "TAKE A BREAK";
  }
  text(s, width/2, height/2);
}

void drawInfo() {
  String s = "";
  textFont(infoFont);
  textAlign(CENTER,CENTER);
  noStroke();
  if (state == 0) fill(255,255,255);
  else if (state == 1) fill(0,255,0);
  else if (state == 2) fill(255,255,0);
  else if (state == 3) fill(255,0,0);
  
  if (leftHandBranches.size() > 0 || rightHandBranches.size() > 0) {
    s = "PLANT YOUR FLOWER FOR OTHERS TO SEE";
  }
  else if (bodyBranches.size() > 0) { // little movement
    s = "TRY MOVING THE LOTUS PLANTS WITH YOUR HANDS";
  }
  text(s, width/2, height/8);
}

void makeGroundBranches() { 
  float leafFatness = random(1, 2);
  int[] leafColorBounds = {0, 196, 140, 200, 0, 0};
  while (groundBranches.size() < MAX_GROUND_BRANCHES) {
    groundBranches.add(new Branch(new PVector(random(20, width-20), height - random(1)*(height/5)), 0.6, leafFatness, leafColorBounds));
  }
}

class Branch {
  PVector pos;
  float percentagePosition;
 
  int numNodes;
  float[] nodeX, nodeY;
  color[] nodeColors;
  float nodeDist; //vertical dist between nodes
  float wiggle; // wonkiness
  
  private float bottomX, bottomY, topX, topY = 0;
  public Float[] modelLine = {0.0, 0.0, 0.0, 0.0};
  
  int numLeaves;
  color[] leafColors;
  float[] leafRotations;
  float[] leafScales;
  float leafFatness;
  
  int[] stemColorBounds = {32, 32, 140, 200, 0, 0};
  int[] leafColorBounds;
  
  float maxHeight;
  float scale = 0.0;
  
  boolean isCollision = false;
  boolean isInFront = true;
  
  Branch(PVector pos, float maxHeight, float leafFatness, int[] leafColorBounds) {
    this.pos = pos;
    this.maxHeight = maxHeight;
    this.leafFatness = leafFatness;
    this.leafColorBounds = leafColorBounds;
    
    this.numNodes = (int)random(21, 35);
    this.nodeDist = (height*maxHeight - 100) / numNodes - random(numNodes/5);
    this.wiggle = random(0.03, 0.07); 
    
    init();
  }
  
  Branch(float percentagePosition, float maxHeight, float leafFatness, int[] leafColorBounds) {
    this.pos = new PVector(width*2,height*2);
    this.percentagePosition = percentagePosition;
    this.maxHeight = maxHeight;
    this.leafFatness = leafFatness;
    this.leafColorBounds = leafColorBounds;
    
    this.numNodes = (int)random(21, 35);
    this.nodeDist = (height*maxHeight - 100) / numNodes - random(numNodes/5);
    this.wiggle = random(0.03, 0.07);
    
    init();
  }
  
  Branch(PVector pos, float leafFatness, int[] leafColorBounds) {
    this.pos = pos;
    this.leafFatness = leafFatness;
    this.leafColorBounds = leafColorBounds;
    
    this.numNodes = (int)random(21, 35);
    this.nodeDist = 0.1;
    this.wiggle = 0;
    
    init();
  }
  
  Branch(float percentagePosition, float leafFatness, int[] leafColorBounds) {
    this.pos = new PVector(width*2,height*2);
    this.percentagePosition = percentagePosition;
    this.leafFatness = leafFatness;
    this.leafColorBounds = leafColorBounds;
    
    this.numNodes = (int)random(21, 35);
    this.nodeDist = 0.1;
    this.wiggle = 0;
    
    init();
  }
  
  void init() {
    nodeX = new float[numNodes];
    nodeY = new float[numNodes];
    nodeColors = new color[numNodes];
    
    nodeX[0] = 0;
    nodeY[0] = 0;
    
    for (int i = 1; i < numNodes; i++) {
      nodeX[i] = nodeX[i - 1] + i * wiggle * random(-10, 10);
      nodeY[i] = -nodeDist * i;
      nodeColors[i] = color(random(stemColorBounds[0], stemColorBounds[1]), random(stemColorBounds[2], stemColorBounds[3]), random(stemColorBounds[4], stemColorBounds[5]));
    }
    
    checkEndpoints();
    
    numLeaves = int(random(9, 21));
    
    leafColors = new color[numLeaves];
    leafRotations = new float[numLeaves];
    leafScales = new float[numLeaves];
    for (int i = 0; i < numLeaves; i++) {
      leafColors[i] = color(random(leafColorBounds[0], leafColorBounds[1]), random(leafColorBounds[2], leafColorBounds[3]), random(leafColorBounds[4], leafColorBounds[5]));
      leafRotations[i] = random(PI, TWO_PI);
      leafScales[i] = random(0.025, 0.030) * (numLeaves + i);
    }
  }
  
  void adjustPosition(Float[] line) {
    pos.x = line[0] + (line[2]-line[0])*percentagePosition;
    pos.y = line[1] + (line[3]-line[1])*percentagePosition;
    
    checkEndpoints();
  }
  
  void checkEndpoints() {
    bottomX = nodeX[0];
    bottomY = nodeY[0];
    topX = nodeX[numNodes-1];
    topY = nodeY[numNodes-1];
  }
   
  boolean checkModelLineIntersection(Float[] otherLine) {
    return checkLineIntersection(modelLine, otherLine);
  }
  
  boolean checkLineIntersection(Float[] line0, Float[] line1) {
    float p0x = line0[0];
    float p0y = line0[1];
    float p1x = line0[2];
    float p1y = line0[3];
    
    float p2x = line1[0];
    float p2y = line1[1];
    float p3x = line1[2];
    float p3y = line1[3];
    
    float s1x, s1y, s3x, s3y;
    s1x = p1x - p0x;
    s1y = p1y - p0y;
    s3x = p3x - p2x;
    s3y = p3y - p2y;
    
    float s, t;
    s = (-s1y*(p0x-p2x) + s1x*(p0y-p2y)) / (-s3x*s1y + s1x*s3y);
    t = (s3x*(p0y-p2y) - s3y*(p0x-p2x)) / (-s3x*s1y + s1x*s3y);
    
    if (s >= 0 && s <= 1 && t >= 0 && t <= 1) return true;
    else return false;
  }
  
  void checkIsInFront(Float[] leftFootLine, Float[] rightFootLine) {
    // drop a line from bottom coord to beyond lower edge of screen
    Float[] checkLine = {modelLine[0], modelLine[1], modelLine[0], float(2*height)};
    // connect the feet nodes of the person
    Float[] footLine = {0.0, max(leftFootLine[1],leftFootLine[3]), float(width), max(rightFootLine[1],rightFootLine[3])};
 
    // if foot line intersects check line, that means that the person is stepping in front of the branch - the person is in front
    boolean hasCollision = checkLineIntersection(checkLine, footLine);
    
    if (hasCollision) isInFront = false;
    else isInFront = true;
  }
  
  void checkIsCollision(Float[] firstLine, Float[] secondLine) {
    isCollision = checkLineIntersection(modelLine, firstLine) || checkLineIntersection(modelLine, secondLine);
  }
  
  void display() {
    pushMatrix();
    translate(pos.x, pos.y);
    scale(scale);
    for (int i = 1; i < numNodes; i++) {
      drawStem(i);
    }
    for (int i = 0; i < numLeaves; i++) {
      drawLeaf(i);
    }
    
    modelLine[0] = screenX(bottomX, bottomY); // modelBottomX
    modelLine[1] = screenY(bottomX, bottomY); // modelBottomY
    modelLine[2] = screenX(topX, topY); // modelTopX
    modelLine[3] = screenY(topX, topY); // modelTopY
    popMatrix();
    
    //drawModelLine();
  }
  
  void drawStem(int i) {
    stroke(nodeColors[i]);
    line(nodeX[i], nodeY[i], nodeX[i - 1], nodeY[i - 1]);
  } 
  
  void drawLeaf(int i) {
    noStroke();
    fill(leafColors[i]);
    
    pushMatrix();
    translate(nodeX[numNodes - i - 1], nodeY[numNodes - i - 1]);
    rotate(leafRotations[i]);
    scale(leafScales[i]/leafFatness, leafScales[i]*leafFatness);
    curve(-50, 50, 0, 0, 100, 0, 150, 50);
    curve(-50, -50, 0, 0, 100, 0, 150, -50);
    popMatrix();
  }
  
  void drawModelLine() {
    stroke(255,255,0);
    line(modelLine[0], modelLine[1], modelLine[2], modelLine[3]);
  }
}

Float[][] getSkeletonLines(SkeletonData _s) {
  Float[][] skeletonLines = new Float[5][4];
  
  // head
  skeletonLines[0] = getBoneLine(_s, Kinect.NUI_SKELETON_POSITION_HEAD, Kinect.NUI_SKELETON_POSITION_SHOULDER_CENTER);
  // left hand
  skeletonLines[1] = getBoneLine(_s, Kinect.NUI_SKELETON_POSITION_WRIST_LEFT, Kinect.NUI_SKELETON_POSITION_HAND_LEFT);
  // right hand
  skeletonLines[2] = getBoneLine(_s, Kinect.NUI_SKELETON_POSITION_WRIST_RIGHT, Kinect.NUI_SKELETON_POSITION_HAND_RIGHT);
  // left foot
  skeletonLines[3] = getBoneLine(_s, Kinect.NUI_SKELETON_POSITION_ANKLE_LEFT, Kinect.NUI_SKELETON_POSITION_FOOT_LEFT);
  // right foot
  skeletonLines[4] = getBoneLine(_s, Kinect.NUI_SKELETON_POSITION_ANKLE_RIGHT, Kinect.NUI_SKELETON_POSITION_FOOT_RIGHT);
  
  return skeletonLines;
}

void drawBones(Float[][] skeletonLines) {
  if (state == 1) stroke(0,255,0);
  else if (state == 2) stroke(255,255,0);
  else if (state == 3) stroke(255,0,0);
  line(skeletonLines[1][0], skeletonLines[1][1], skeletonLines[1][2], skeletonLines[1][3]);
  line(skeletonLines[2][0], skeletonLines[2][1], skeletonLines[2][2], skeletonLines[2][3]);
  line(skeletonLines[3][0], skeletonLines[3][1], skeletonLines[3][2], skeletonLines[3][3]);
  line(skeletonLines[4][0], skeletonLines[4][1], skeletonLines[4][2], skeletonLines[4][3]);
}

Float[] getBoneLine(SkeletonData _s, int _j1, int _j2) {
  noFill();
  stroke(255, 255, 0);
  Float[] boneLine = {width*2.0, height*2.0, width*2.0, height*2.0};
  if ((_s.skeletonPositionTrackingState[_j1] != Kinect.NUI_SKELETON_POSITION_NOT_TRACKED) && (_s.skeletonPositionTrackingState[_j2] != Kinect.NUI_SKELETON_POSITION_NOT_TRACKED)) {
    boneLine[0] = _s.skeletonPositions[_j1].x*width;
    boneLine[1] = _s.skeletonPositions[_j1].y*height;
    boneLine[2] = _s.skeletonPositions[_j2].x*width;
    boneLine[3] = _s.skeletonPositions[_j2].y*height;
  }
  return boneLine;
}

void appearEvent(SkeletonData _s) {
  if (_s.trackingState == Kinect.NUI_SKELETON_NOT_TRACKED) return;
  synchronized(bodies) {
    synchronized(skeletonLines) {
      if (bodies.size() < 1) {
        bodies.add(_s);
        Float[][] currentSkeletonLines = getSkeletonLines(_s);
        skeletonLines.add(currentSkeletonLines);
      }
    }
  }
}

void disappearEvent(SkeletonData _s) {
  synchronized(bodies) {
    synchronized(skeletonLines) {
      for (int i = bodies.size()-1; i >= 0; i -= 1) {
        if (_s.dwTrackingID == bodies.get(i).dwTrackingID) {
          bodies.remove(i);
          skeletonLines.remove(i);
        }
        else if (bodies.get(i).dwTrackingID == 0) {
          bodies.remove(i);
          skeletonLines.remove(i);
        }
        else {
        }
      }
    }
  }
}

void moveEvent(SkeletonData _b, SkeletonData _a) {
  if (_a.trackingState == Kinect.NUI_SKELETON_NOT_TRACKED) return;
  synchronized(bodies) {
    for (int i = bodies.size()-1; i >= 0; i -= 1) {
      if (_b.dwTrackingID == bodies.get(i).dwTrackingID) {
        bodies.get(i).copy(_a);
        break;
      }
    }
  }
}