Face Tracking and Movement Following Drone

We all must have seen a drone and some of us might have even operated one. Most of the drones available in the market are operated manually either with the help of an RC remote or by using a phone over a WiFi connection. Operating a drone this way can be a bit tricky, especially when there are lots of controls and buttons. 

To make the experience of flying a drone even more fun, I have decided to do something different by building a smart drone control system that can be operated with the help of our facial gestures. If we want the drone to take footage of a particular face and track it, then we can use this system.

Here I have used a DJI Tello drone that is programmed and controlled using Python and its SDK. So the components you need here are only a Tello drone and a PC or laptop. That’s it.

- Advertisement -

Prerequisites 

As I have used the Linux OS, the commands and the setup instructions are based on Linux. You can run Linux on Windows using VirtualBox, an open-source hypervisor that supports a large number of guest operating systems. 

First of all, set up the Python environment in the OS. Then install the OpenCV, dlib, NumPy and face recognition Python modules using the following commands: 

- Advertisement -

sudo pip3 install python-opencv

sudo pip3 install Face_recoginition 

sudo pip3 install numpy

After installing the above modules, install the Python modules that enable connection with the Tello drone to control it. For this, there are various Python modules available but here we will use the Tello Python to control the Tello drone. 

NOTE:- Before making a drone control system, check some functions and commands, which we will use for controlling the drone.

The Tello Python can connect with the Tello drone, allowing different functions to be performed such as:

• Launching the drone using drone.takeoff()
• Moving the drone forward within a certain distance (in cm) using drone.forward(cm)
• Rotating the drone to 90 degrees in a clockwise direction using drone.cw(90)  
• Flipping the drone upwards, backwards, left or right using drone.flip(‘l’) 
• Creating a live stream through drone camera using drone.streamon()  
• Landing the drone using drone.land()  

To control the height and left or right movements of the drone, use your facial gestures but for take off, use the automated takeoff function on startup. To land the drone and to stop the code from running, use the keyboard key ‘Q’. 

Refer to the same code for face recognition as given in several of my other projects including Face Recognition-Based Car Lock, AI Robot, Smart Door and many more. By implementing this code, you can control the drone using your face movements.

Coding 

In addition to including the same code for face recognition, import the Tello drone libraries and modules (mentioned in the Prerequisites section). After that, create two variables that will store the face location coordinates, which will facilitate the calculation of the face movements and navigation of the drone. Initialise drone.streamon() in the code along with drone.takeoff().

In the while loop of the face recognition code, add another code that analyses the directions of the face movements. Check the current location of the face and using the if condition compare the present face location with the previous face location. If they do not match, then it means that the face has moved to a new location. If the new location has a value greater than or smaller than the previous location, use it to determine the directions of face movement. 

Use the upward, downward, left or right direction movements of your face to move the drone. By using the left-point and top-point position values, one can determine and compare the face position and movement direction for controlling the drone in the same direction.

Suppose the previous left-position = 30 in the x-coordinate and the new left-position = 40 in the x-coordinate. Since the new left-position is greater than the previous left-position, you have to move your face towards the new direction i.e. right. This will in turn move the drone in the same direction. Similarly, the drone will move in other directions depending on the change in coordinates.

Testing

Switch on the Tello drone and then connect the device on which you will run the above given code. I am using Raspberry Pi, which is connected to the Tello WiFi for running the code. After a few seconds, the connection is made. On running the code, the video of the face appears. By moving your face within the camera frame upwards, downwards, left or right directions you can control the drone.

Congratulations!! You have successfully built a facial gesture operated drone.

Code