How to Control Robots With a Game Controller and Arduino

Ian Buckley 01-03-2018

Arduinos and similar compatible boards are one of the go to devices for DIY tinkerers everywhere. Whether you are a beginner who is just getting started with Arduino or someone who has already been putting them to use in your life, they provide a platform for countless awesome projects.


Today we will be exploring a creative way to control a servo using Processing, and an Xbox360 controller. If you are already pretty game development savvy, you might be interested in our Custom Game Controller tutorial, which uses Unity.

This tutorial will assume a little prior knowledge, if this is your first foray into Arduino fiddling, you might find our Arduino guide useful here. Similarly, if this is your first time using Java it can be a little confusing. While Processing uses a simplified version of the platform, these Java concepts and tips might still help.

What You Need

control robots with game controller and arduino

  • 1 x Arduino. We are using an UNO today.
  • 1 x hobby servo. Anything that will work with the Arduino pins.
  • 1 x wired Xbox360 controller. Although this will technically work with almost any controller.
  • Several hookup wires.

In addition to these things, you’ll also need to download Processing and the Arduino IDE from their respective websites.

Preparing the Arduino

control robots with game controller and arduino


First we need to attach our servo. The wiring colours can vary here, but as a general rule red attaches to the 5v pin, and brown or black attach to the GND pin. The data line, which is usually yellow or orange, attaches to pin 10.

Check your wiring and connect the Arduino to the computer. Open up the Arduino IDE.

control robots with game controller and arduino

Open up the StandardFirmata sketch located at File > Examples > Firmata > StandardFirmata. This sketch sets up the board for external control over the serial port, and is the same one we used in our article on controlling Arduino with Python How to Program and Control an Arduino With Python Sadly, it is impossible to directly program an Arduino in Python, but you can control it over USB using a Python program. Here's how. Read More . Upload the sketch to the board.


If the upload fails, check you’ve selected your correct board and port details in the Tools menu.

Our Arduino is ready to go!

Setting Up Processing

Open up processing, you’ll be greeted with a blank sketch. Before we do anything here we will need to install a few libraries. Head to the Sketch menu and select Import Library > Add Library. This will bring up the Contribution Manager which will look familiar to any Arduino users out there.

control robots with game controller and arduino


We need to install three libraries to make this work. First up is the Game Control Plus library. This is what will allow us to use our game controller with Processing. Use the search window to find it, and click install in the bottom right hand corner. Game Control Plus needs another library installed for its configuration tool, so let’s get it now. Search for the G4P library and install it too.

Finally, we need the Arduino (firmata) library. You guessed it, search for it, and click install. With these things installed we are ready to get on with testing that everything will work. We are working with Windows 10 today, but processing is available for most platforms, including Raspberry Pi. Imagine the possibilities!

Testing the Arduino

Before we dive into creating a custom sketch, let’s test the Arduino and Servo with Processing. Open File > Examples, and select ArduinoServo from the Contributed Libraries/Arduino (firmata) folder. We’ll use this to test our servo, but first we may need to change a couple of things.

Scroll down through the sketch and find this line:


If it is commented out, remove the two slashes before println(Arduino.list());, and save the sketch. Run it, by clicking the play icon, and keep an eye on the console at the bottom. This will list everything attached to your COM ports.

control robots with game controller and arduino

In my case, my Arduino was on COM 8, which was the third port listed here. This is important as the code in the line below has an Array whose value determines which COM port to use.

control robots with game controller and arduino

We need to change this to reflect our COM port. For me, it was the third position, or index number 2:

arduino = new Arduino(this, Arduino.list()[2], 57600);

We need to make a couple of other small changes to this code to test it. Scroll down to where the Arduino pins are setup and comment out one of the lines here. Change the other one to Pin 10.

//arduino.pinMode(4, Arduino.SERVO);
arduino.pinMode(10, Arduino.SERVO);

We need to do the same thing in the Draw() method:

arduino.servoWrite(10, constrain(mouseX / 2, 0, 180));
// arduino.servoWrite(4, constrain(180 - mouseX / 2, 0, 180)); 

Save the sketch, and run it. You should be able to move your servo by moving your mouse back and forth across the window the program generates.

control robots with game controller and arduino

If it doesn’t work for you, check your Servo wiring, and check you have the right array position for your COM port. Once you know the Arduino is talking nicely with Processing, it’s time to move on.

Configuring the Controller

The Game Control Plus library we are using comes with a powerful configuration too. Make sure your controller is plugged in, open the Configurator example project, and run it. You will get a menu like this:

control robots with game controller and arduino

Click on your controller name, and a much larger configuration window will pop up.

control robots with game controller and arduino

This may look fairly daunting, but it’s designed to be as simple as possible. On the left side fill out the first key with the name you want as a variable. This variable will control the position of the servo, so I’m going to call it servoPos.

In the box next to it you can give a brief description of what it does. Now pick up your controller and move the stick you wish to use with your servo. A little experimentation shows that the right thumbstick corresponds with the X Rotation box. Drag a line between the servoPos variable, and this box.

control robots with game controller and arduino

Now we need to save our configuration as a data file. In the top right of the window, fill out the Device role field and the Filename field.

The filename is important, as you’ll be using it in your code. I’m keeping it simple by calling it xbs. Click Verify then Save. This writes a file with instructions for our controller which we can use later.

control robots with game controller and arduino

Preparing the Custom Sketch Folder

Let’s set up our working folder. Open up a blank processing sketch, and save it under whatever name you like. This will create a directory for it in the save location.

Now navigate to Documents/Processing/libraries/GameControlPlus/examples/Configurator and copy the folder labelled data. This folder contains the configuration file we just created. Navigate to the directory of your newly saved blank sketch, and paste the data folder.

control robots with game controller and arduino

Creating the Custom Sketch

Now everything is in place and we can start making a sketch using our two libraries. We’ll go through this step by step, but you can download the full sketch and data folder if you want to jump ahead. Note that you may still need to modify the code to reflect your Arduino COM ports.

Begin by importing all the libraries we will need:

import processing.serial.*;
import org.gamecontrolplus.*;
import org.gamecontrolplus.gui.*;
import cc.arduino.*;
import org.firmata.*;

We also need to declare our ControlDevice, I/O, and Arduino, along with a float to hold values from our thumbstick:

ControlDevice cont;
ControlIO control;
Arduino arduino;
float thumb;

Our setup() method creates a small window, an instance of the controller, and matches the device with our config file. This is where it is important to get the filename of our configuration data file correct:

void setup() {
  size(360, 200);
  control = ControlIO.getInstance(this);
  cont = control.getMatchedDevice("xbs");

  if (cont == null) {
    println("not today chump"); // write better exit statements than me
  // println(Arduino.list());
  arduino = new Arduino(this, Arduino.list()[2], 57600);
  arduino.pinMode(10, Arduino.SERVO);

We also check if there is no applicable controller at this stage, and quit out of the program if needs be. While the window created with size() is not needed, it will give us some feedback later as to whether we are getting useful values from our controller. We also initialise our Arduino and pin here just like we did while testing.

Now we create a little method to grab the input value from our controller, and map it to values our servo will be able to use:

public void getUserInput() {
  thumb = map(cont.getSlider("servoPos").getValue(), -1, 1, 0, 180);

This one line of code uses our data file to get our named control servoPos, which is linked to the right thumbstick of the controller, and read values from it. It then maps the values and stores the value in our thumb float variable.

Right now this code never gets called, we’ll fix that now.

void draw() {
  arduino.servoWrite(10, (int)thumb);

The draw() is similar to the loop() method in the Arduino IDE. Every frame, it calls the getUserInput() method and updates the thumb value. It uses this value to change the red value of the background() giving us a visual indicator of the change in value. It then writes this value to the servo using the arduino.servoWrite() function. Note that we have to cast thumb as an integer value as the servoWrite function takes two integers (pin number, and angle) as its arguments.

Check your code for errors, save it, and click run. After a slight delay to initialise the Arduino, it should look like this:

control robots with game controller and arduino

Control With Game Controller and Arduino: Finished!

This project was in many ways quite in depth for the inexperienced coder Learn With Coding Projects: 9 Udemy Courses for the Beginner Programmer The real difficulty is finding good programming tutorials and courses that not only teach you the necessary skills, but do so with practical projects. That's where Udemy steps in. Read More , despite the fantastic libraries available to help us. What it represents is a new way to think about controlling robots, and any other devices you build.

This project would go along perfectly with our guide on building a Laser Turret Pew Pew! How To Build A Laser Turret With An Arduino Are you bored? Might as well build a laser turret. Read More , giving you full control over it. You could set up a Piezo buzzer like in our Simple Arduino Alarm How To Make a Simple Arduino Alarm System Detect movement, then scare the heck out of an intruder with a high pitched alarm sounds and flashing lights. Does that sound fun? Of course it does. That's the goal of today's Arduino project, suitable... Read More tutorial and use your controller to change the pitch of the buzzer or color of the lights.

Or you could, well, build a massive robot and take over the earth. As long as you had a USB cord long enough!

Related topics: Arduino, Game Controller, Robotics.

Affiliate Disclosure: By buying the products we recommend, you help keep the site alive. Read more.

Whatsapp Pinterest

Leave a Reply

Your email address will not be published. Required fields are marked *

  1. Soren Anderson
    July 14, 2020 at 1:19 pm

    Hey Ian, This was a tremendous project and what I've been looking for to build out my own flight control systems.

    I have questions: If I'm using a Xbee to communicate ground station (just an xbox controller, arduino, and lcd monitor), what would have have to do? Do I need the compiler on the arduino to translate the Game Controller Plus, or do I just need that data file?

    Have you done this yourself or can you point me to a project that has? Maybe you'd like to collab? Thanks!

  2. Riley Hollis
    June 18, 2019 at 1:02 pm

    how would i do connect this wirelessly to an esp8266?

  3. Ian Buckley
    April 18, 2019 at 6:32 pm

    I'm honestly not sure. Do you get any errors in the console? It's possible that feeding impossible values to the either the window or the servo causes a crash.

  4. Alex LaRuffa
    April 15, 2019 at 3:29 pm

    How would the code change if I were to use a motor instead of a servo? How would the code change if I were to use a button on the Xbox controller for servo control?

    • Ian Buckley
      April 18, 2019 at 6:49 pm

      Ok, super quick - I'm on phone and making dinner for the family so no apologies for mistakes!

      Firstly to use a motor you would need a motor shield, something with the L298N chip or similar.

      Instead of sending a value of 0-180 on the servo pin, motors use three separate pins, one for each direction, and one for speed - if you get stuck, follow a basic tutorial on using a motor shield first to get the hang of the code, then modify the above code to change the motor direction and speed in accordance to the mapped "thumb" value.

      Changing the servo to use a button takes two steps. First, set up the button on the controller - the same Processing Sketch I used to set up the thumb stick will help with this. Next, you need to change the code to move the servo to a specific place when the button is held down, rather than mapping the thumb stick - buttons are Boolean (not always but it's easier to think like that in this instance), so you need to think about what you want the servo to do when the button is on, and when it is off, and add logic (probably if statements) to make it so what you want.

      Ok, as promised, rushed but gotta go, hope that helped!

      • Jayee
        January 13, 2020 at 1:13 am

        I am doing the same project, but using xbox to control dc motors as well. I am able to read values from xbox. However Processing is not outputing any values to any of my arduino pins. I followed your exact same steps. I simply use arduino.digitalWrite() to test to see if the value got set to HIGH but it didn't. I uploaded the standardFirmata onto my arduino. Do you have any idea on this issue?

  5. Sebastian
    April 24, 2018 at 11:36 pm

    When I move my mouse or joystick too fast and try to move the servo too fast, the program stops working unless I close the window and run the program again. Is this normal? And if not, how would I fix it? Thank you!