IoT On The Eyes: To Control Home Appliances

Have you ever thought of controlling electric/electronic home appliances by simply looking at them? Could happen in movies but seems impossible in real life.

Well, no more. Here is perhaps world’s first device that lets you switch on or switch off any electric/electronic appliance with the blink of your eyes. This cool IoT device would also help specially-abled people to independently control such appliances.

Full Video Explaination:

- Advertisement -

Coding

The device needs to recognise the appliances that have to be controlled through commands sent from the eye. Therefore, the following code permits capture of real-time videos for object detection and toggling an appliance on or off.

OpenCV is used to capture video from the camera and TensorFlow (TF) to recognise the home appliances being looked at.

- Advertisement -

Install Python and the required modules in Raspberry Pi using following commands:

sudo pip3 install python-opencv
sudo pip3 install tensorflow
sudo pip3 install keras
sudo pip3 install gpiozero
Sudo pip3 install pyserial
git clone –depth 1 https://github.com/
tensorflow/models.git

If you face a problem in installation of TensorFlow and OpenCV, or get an error due to the version of TensorFlow, follow the instructions for installation of TensorFlow and object detection given in TensorFlow website.

Next, clone the object detection library. Then in the test folder, create a file containing the list of appliances and save it as eyeiot.pbtxt

To create the IoT object detection and controlling functionalities, copy the code for object detection to a new file. Make sure to import a few more modules and libraries to the code such as gpiozero to control the pin inputs and outputs and pyserial to send the command to Pico to control the relay and thus the appliances.

We use pyserial module to connect to RPi Pico using Bluetooth HC 05 attached with RPi Pico and built-in Bluetooth of Raspberry Pi 4. So, in the code import the py serial and set the baud rate and serial name of Bluetooth serial port

Finally, set the file name eyeiot.pbtxt in the code so that it can detect which appliance you wish to control from the given list of appliances (whether it is refrigerator, TV, toaster, or any other). After doing so, obtain the result where the strings contain the list of detected object names on the video.

The code will search various sub-string names in the string format results. These sub-strings contain the names of the appliances you wish to control. A few conditional statements will check the availability of that string, that is, the presence of an appliance.

Suppose you wish to operate a TV. The code will detect its presence when you look at it. With the help of an if condition, time will be counted while you are looking at it. After five seconds the TV will switch on or switch off.

Raspberry Pi Pico coding

We need to create code for Raspberry Pi Pico so that it may connect to the main Raspberry Pi and control the device wirelessly we are looking at. We will use micropython for Raspberry Pi Pico. We use USART to communicate with the Bluetooth HC 05 module and then using if condition check the commands to toggle the switch of connected appliances. Save the code as main.py to Raspberry Pi Pico.

Connection

Connect HC05, Pico, and the relay module as shown in Fig. 7. Connect the relay’s COM pin to one wire of the electronic appliance and its normally-open pin to the live wire carrying AC voltage from the power supply. Connect the other wire of electric appliance to neutral wire of power supply. (Warning: Be care careful while working with AC power supply as wrong connection might cause electric shock and short circuit.)

You can add one more relay to 25-pin connector for phone and laptop charging.
Next, connect Raspberry Pi Pico to 5V DC adaptor and connect the Raspberry Pi 4 with a camera mounted on eyeglass with 5V battery and run the code iotoneye.py on Raspberry Pi 4. At EFY we included TV in the code for testing. When you look at TV for five seconds, it automatically turns on/off. Likewise, you can switch on/off any device.

Download Source Code

Ashwini Kumar Sinha is a technology enthusiast at EFY

This project was published online on 25th March 2021 and was republished in print on October 2021.