New Facial Recognition Smart Glasses For Visually Challenged Person

Ashwini Kumar Sinha

2109
Advertisement
Change Video Language

In this project, we will be making a prototype of a smart eyeglass that can help a visually
challenged person to recognize the person in front of him and learn about obstacles ahead.
This will be enabled by face recognition and distance detection features.

Bill of Material

Bill of material

You will also need an earphone but since most of us have it, I haven’t mentioned it in the
above list.

Prerequisite setup
First, open RPi SSH or Terminal and install the required libraries and modules.
Then install the espeak module, followed by the espeak library for Python.
Follow the snippet below for installation (Refer Fig 1,2).

installing espeak
Fig 1. Installing espeak
installing espeak
Fig 2. Installing espeak for python
Advertisement

Now install the ‘opencv’, ‘numpy’, ‘dlib’ library. You might need to swap the
memory for installation. Follow the instruction for installation of opencv and dlib
available on the internet.

After that install the face recognition library for python 3 using the following
command in the terminal.

‘pip3 install face recognition’

As we have installed our required library, let’s start coding.

Code

For this project we will make two different codes in Python3, one to tell the distance of
obstacles ahead and other for face recognition. The first code measures the distance of
obstacles and alert the user using voice output. We can also add a vibration motor for haptic
feedback.
Code 1: Open the Python3 IDE to start coding. Here we need two modules – time and
espeak. We will import both the modules in code. After that we created a distance function
where we import the GPIO and set the pins for distance sensor. Next we created a while
loop that continuously checks the distance. In this while loop we also added an if condition
to check the distance between the user and the obstacle. If it detects an object close to the
user, it will automatically inform the person by giving audio and haptic feedback. (Refer fig
3, 4).

Distance sensor code
Fig 3. Python code distance sensor
Distance sensor code
Fig 4.Python Code obstacle detection.

Code 2: This is to recognise the person in front of the user (known or unknown). Here in this code we will import 3 modules: face recognition, cv2 and numpy. We have to create different arrays for known faces and their names. We must write the image file name for the face recognition of that member (Refer Fig 5,6). 

Code
Fig 5.Python code face detection
Python code
Fig 6.

In the next part of code, we will try to match the face in camera video with the array of known faces. If the face matched, then the code will run the espeak synthesizer to call the name of that person using the syntax ‘espeak.synth ()’ as in the pic below.(Refer fig 7).

Python code Smart Face Recogination
Refer fig 7.

Note :- You can Change the output language  to hindi and voice to different modes for example wisher mode, Male voice , and female voice by uncommenting lines in code.(Refer fig 8).

Python code
Fig 8. Espeak Different Language and voice

After all is done, save the code in the face recognition folder.

Download Code

Connection

Since we have created the code, now it’s time to connect all the components. 

Insert the camera ribbon in camera module and solder the wires of ultrasonic sensor as illustrated below.

Raspberry Pi Ultrasonic Sensor
5V pin VCC
GND GND
BCM Pin 27  TRIG
BCM PIN 22 ECHO

 

After the connection, set all sensors and components as in pic below.

Smart Glasses
Fig 9.
Smart Glasses
Fig 10.

Testing

Now power the Raspberry Pi using battery or power bank.  Then open the Pi desktop window in remote desktop or VNC (If you do not know how to enable VNC and remote desktop on Pi follow the instructions online.) Next run the distance.py and facerecogination.py code and you are ready to go. When any known or unknown person come in front of the user (visually challenged person), it will detect the face and tell about them. If the user comes closer to any obstacles like wall or any other object, the device will alert about it to avoid accident.      

Note: – This is the first version of the project, we will update it soon with more features to
enable detection of colour and ambient light. The update may come with optical character
and sign board recognition feature as well.
Note: –During the installation of cv2 , dlib, and face recognition library you might need to swap the memory of Raspberry Pi do it carefully.If you face any error or problem during the installation of library  refer to online guide of that python library.

 

Advertisement


SHARE YOUR THOUGHTS & COMMENTS

Please enter your comment!
Please enter your name here