Designing IoT based Face Recognition AI Robot

In this project, we are going to learn how to design a smart AI robot that can recognize a person’s face, can recognize that person’s voice and thus receive various commands for welcoming guests or for security purposes. One can also preview live streaming with a camera through face recognition.

Note: This is the first version of the robot. We uploaded the updated version.

Components Required

For this AI robot, we need the following components. 

- Advertisement -

• 2 Servo Motors
• 2 OLED SSD1306s
• 2 Arduino boards 
• Raspberry Pi 4 board
• Raspberry Pi camera
• Amplifier
• Bluetooth HC 05
• Bluetooth Audio receiver
• 3 thin metallic rods (7cm long) 
• A hard-cardboard or wooden base to mount the robot head
• USB data cable
• 5V power adapter
• Connecting wires
• Last but not least, a robot face mask (Here we are using an open source INMOOVE robot head).

NOTE: As the sensors and parts are customizable, the above bill of materials can vary.

Also Check: Unveiling Our Open-Source Facial Expression Recognition Technology

- Advertisement -

Assembling the AI Robot’s Face

This may vary according to the different shapes and sizes of the robot head design. Here, I am using the Open Source INMOOV robot head that has been created using a 3D printer. You can even 3D print your own face and use it as a robot head, or get various 3D printed robot heads from thingiverse.com. 

NOTE:- Before assembling the OLED display for the robot’s eyes, first solder it to make the proper connections (as explained later).

Make sure to securely screw the OLED displays onto the eye piece of the robot head as shown in Fig. 1

After this, mount the Raspberry Pi camera (connected to the Raspberry Pi board) carefully beside one of the OLED displays. Attach a servo motor near the mouth of the robot head. This will allow the robot’s jawline to open and close (See Fig. 2, 3)

Supporting the Robot head

First, firmly fix another servo motor on a cardboard or wooden base with the help of a screw or hot glue. This will provide up and down movement to the robot head.

To make a sturdy support, screw three thin metallic rods in such a way that they surround the servo motor in a triangular shape. Make to sure to connect one end of a metallic rod with the shaft of motor. (See Fig. 4, 5, 6).

Now the basic assembly portion is done. The final AI Robot head will look like this (See Fig. 7).

Now, let’s move on to the coding part.

Prerequisite

Before we begin, go to the Library Manager of Arduino IDE and install the following libraries:

• Adafruit GFX 
• SSD1306 OLED

Similarly, install the following modules in the Raspberry Pi for Python 3 environment.

• Numpy
• Scipy
• Open cv
• Cv2
• Face recognition 
• Espeak
• Matplot
• Keras

To install these libraries, follow the library installation instructions available at the documentation folder of each library.

Coding for the Robot’s Eyes

We will add the above Arduino libraries into the code using ‘include’ function and then insert the bitmap hexadecimal code for the eyes (See Fig. 8).

Next, we create a loop function to call the above bitmap codes to preview them on the OLED display.  (See Fig. 9).

Coding for Face Recognition 

This is to recognize the person in front of the robot (known or unknown). In this code, we will import 3 modules: face recognition, cv2 and numpy. We will create different arrays for recognising faces and names. Make sure to write the image file name of that member for correct face recognition. (Refer fig 10 and 11).

In the next part of code, we will try to match the face that has been captured by the camera with the array of known faces. If the face matches, then the code will run the espeak synthesizer to speak the person’s name using the syntax ‘espeak.synth ()’ as in the pic below (See Fig.12)

Circuit Connection

OLED to Arduino Connection

To get audio output with the help of a Raspberry Pi, you can follow the two processes below:

•  Connect any small speaker via the TTRRS (aux) audio output of Raspberry Pi to TTRRS(Aux) of speaker, or
• Use a small speaker with an amplifier.

Now connect one end of Raspberry Pi camera’s ribbon cable to the Raspberry Pi camera input present on the board. Then power the Arduino Mini connected with the OLED display via 5V pin of Raspberry Pi. 

Now our AI Robot is ready to work.

Testing

After booting the Raspberry Pi, open the face recognition script that we have made and run that script. Now, on the OLED display, you can see the robot’s eyes move. Whenever you will go in front of the camera, the robot will recognize your face and will say your name.

Download Source Code

If you found this IoT project useful, then you may also like Top 30+ IoT Projects