Processing is a powerful tool which allows the creation of art through code. It is the combination of a Java library for working with graphics, and an integrated development environment (IDE) which allows you to write and run code easily.
There are many graphics and animation beginner projects which use Processing, but it is also capable of manipulating live video.
Today you'll be making a live video slideshow of different effects controlled by the mouse, using the Processing video library. As well as flipping the live video, you'll learn to resize and color it, and how to make it follow the mouse cursor.
Project Setup
To begin, download Processing and open a blank sketch. This tutorial is based on a Windows system, but it should work on any computer with a webcam.
You may need to install the Processing Video library, accessible under Sketch > Import Library > Add Library. Search for Video in the search box, and install the library from The Processing Foundation.
Once installed, you are ready to go. If you want to skip the coding, you can download the complete sketch. It is much better to make it yourself from scratch, however!
Using a Webcam With Processing
Lets begin by importing the library, and creating a setup function. Enter the following into the blank Processing sketch:
import processing.video.*;
Capture cam;
void setup(){
size(640,480);
cam = new Capture(this, 640, 480);
cam.start();
}
After importing the video library, you create a Capture instance called cam to store the data from the webcam. In setup, the size function sets up a 640x480 pixel sized window to work in.
The next line assigns cam to a new instance of Capture, for this sketch, which is the same size as the window, before telling the camera to turn on with cam.start().
Don't worry if you don't understand every part of this for now. In short, we have told Processing to make a window, find our camera, and turn it on! To display it we need a draw function. Enter this below the code above, outside of the curly brackets.
void draw(){
if (cam.available()){
cam.read();
}
image(cam,0,0);
}
The draw function gets called every frame. This means that many times each second, if the camera has data available you read the data from it.
This data is then displayed as an image, at the position 0, 0, which is the top left of the window.
Save your sketch, and press the play button at the top of the screen.
Success! The data stored by cam is correctly being printed to the screen every frame. If you are having problems, check your code thoroughly. Java needs every bracket and semi-colon in the right place! Processing can also require a few seconds to access the webcam, so if you think it isn't working wait a few seconds after launching the script.
Flipping the Picture
Now that you've got a live webcam image let's manipulate it. In the draw function, replace image(cam,0,0); with these two lines of code.
scale(-1,1);
image(cam,-width,0);
Save and rerun the sketch. Can you see the difference? By using a negative scale value, all of the x values (the horizontal pixels) are now reversed. Because of this, we need to use the negative value of the window width to position the image correctly.
Flipping the image upside down requires just a couple of small changes.
scale(-1,-1);
image(cam,-width,-height);
This time, both the x and y values are flipped, turning the live camera image upside down. So far you've coded a normal image, a horizontally flipped image, and a vertically flipped image. Let's set up a way to cycle between them.
Making It Cycle
Instead of rewriting your code every time, we can use numbers to cycle through them. Create a new integer at the top of your code called switcher.
import processing.video.*;
int switcher = 0;
Capture cam;
We can use the value of switcher to determine what happens to the camera image. When the sketch starts, you give it a value of 0. Now we can use logic to change what happens to the image. Update your draw method to look like this:
void draw(){
if (cam.available()){
cam.read();
}
if(switcher==0){
image(cam,0,0);
}
else if(switcher == 1){
scale(-1,1);
image(cam,-width,0);
}
else if(switcher == 2){
scale(-1,-1);
image(cam,-width,-height);
}
else{
println("Switcher = 0 again");
switcher = 0;
}
}
Now, all three variations of the code will trigger depending on the value of switcher. If it doesn't match one of our if or if else statements, the else clause gets reset to 0. Logic is an important beginner skill to learn, and you can find out about them and a lot more with an excellent YouTube Programming Tutorial!
Using the Mouse
Processing has built-in methods for accessing the mouse. To detect when the user clicks the mouse, add the mousePressed function at the bottom of your script.
void mousePressed(){
switcher++;
}
Processing listens for any mouse clicks and interrupts the program to carry out this method when it detects one. Every time the method gets called, the value of switcher gets bigger by one. Save and run your script.
Now, when you press the mouse button, it cycles through the different orientations of videos, before returning to the original. So far you have just flipped the video, now let's do something a little more interesting.
Adding More Effects
Now, you will code a four-color live image effect similar to the famous Andy Warhol works of art. Adding more effects is as simple as adding another clause to the logic. Add this to your script between the last else if statement, and else.
else if(switcher == 3){
tint(256, 0, 0);
image(cam, 0, 0, width/2, height/2);
tint(0, 256, 0);
image(cam, width/2, 0, width/2, height/2);
tint(0, 0, 256);
image(cam, 0, height/2, width/2, height/2);
tint(256, 0, 256);
image(cam, width/2, height/2, width/2, height/2);
}
This code uses the image function to create four separate camera images in each corner of the screen and to make them all half sized.
The tint function adds color to each camera image. The numbers the brackets are red, green, and blue (RGB) values. Tint colors all the following code with the chosen color.
Save and play to see the result. Try changing the RGB numbers in each tint function to change the colors!
Making It Follow the Mouse
Finally, let's make the live image follow the mouse position using helpful functions from the Processing library. Add this above the else part of your logic.
else if(switcher==4 ){
image(cam, mouseX, mouseY, width/2, height/2);
}
Here, you are positioning the image from your camera at mouseX and mouseY. These are built in Processing values which return which pixel the mouse is pointing at.
That's it! Five variations of live video through code. However, when you run the code, you'll notice a couple of problems.
Finishing Up the Code
The code you've created so far works, but you'll notice two issues. Firstly, once the four-color variation shows, everything afterward is tinted purple. Secondly, when you move the video with the mouse, it leaves a trail. You can fix it by adding a couple of lines to the top of the draw function.
void draw(){
tint(256,256,256);
background(0);
//draw function continues normally here!
At the start of each frame this code resets the tint color to white, and adds a background color of black to stop the video leaving trails. Now when you test the program, everything works perfectly!
Webcame Effects: Art From Code
Processing is very powerful, and you can use it to do many things. It's an excellent platform for making art with code, but it is equally suited to controlling robots!
If Java isn't your thing, there is a JavaScript library based on Processing called p5.js. It's browser-based, and even beginners can use it to create fantastic reactive animations!
Image Credit: Syda_Productions/Depositphotos