Emotion-sensing technology brings to life a new design approach that will include more complicated tasks than simply creating a visual design. It will combine monitoring and analysis of behaviour patterns, measuring actions, and noting facial expressions, voice intonation and body language, among others.
Reading and understanding human emotions is an integral part of the human civilisation. Women are supposed to be better than men at detecting emotions, especially fear and disgust. A human can deal with any other human if the former can learn from the latter’s emotions carefully. Traders, godmen, fortune tellers and the like have always been masters of understanding emotions.
Sometimes, humans hide their emotions from others, so in some serious cases they even have to be put under lie-detection tests. Close tracking of Internet activities (comments, likes, tags, recommendations, photos and videos), content of their social networks and online interactions with other users together enable understanding of their personalities and behaviour.
Electronics sensing technology has come a long way in understanding human behaviour and emotions in detail and with certainty. The technology is introducing the possibility of recording various reactions, emotions or feelings of a person at a given moment. Further, the technology is changing the way it perceives human emotions and, in the near future, we will not only be able to read the emotions of a person but will be able to look into the future with the help of electronics.
Emotion-sensing technology also brings to life a new design approach that will include more complicated tasks than simply creating a visual design. It will combine monitoring and analysis of behaviour patterns, measuring actions, and noting facial expressions, voice intonation and body language. Smart devices will be able to assess the meaning behind certain emotions and respond accordingly.
Emotion-sensing technology can be applied in such areas as educational and diagnostic software, driverless cars, personal robots, pervasive computing, sentient virtual reality, video games, affective toys and consumer electronics devices. This article discusses the state of progress of the electronics-based emotion-sensing technology.
Technology focusing on emotions
Skin-conductance sensors are widely used by psychologists and therapists in clinics, hospitals and so on. The Internet of Things (IoT) is implementing emotional ingredients in wristbands, too. Sensor technology allows gathering of data on heart rate, blood pressure and temperature to define an individual’s emotional state.
Such emotion-based sensors have a wide range of applications and are easy to use at a relatively low cost. Smart watches and health wearables have already created a foundation for this technology that helps manage daily tasks while avoiding potential health issues and other problems. With such modern smart devices, users can track and evaluate their physical condition, access their reactions to stress-inducing situations, and learn to manage stress and anxiety better.
These devices give instructions to users to practice mind-controlling techniques or breathing exercises, or to turn on relaxing music to help calm them down. Emotion-sensing technology and software supported by emotional artificial intelligence (AI) can read and analyse not only skin conductance, breathing and heart rate, but also eye movements, facial expressions, changes in voice and more. These do not necessarily require installing expensive hardware, but rather just some recognition software or additional code for computers or smartphones.
Growth of emotional intelligence technology is going to produce a number of focus areas that will sense emotions such as:
- Reflecting on a personal emotional experience
- Enabling actions and reactions to emotions
- Facilitating communication
- Turning emotions into experiences
Development highlights
Here are some interesting examples of the applications of emotion-sensing technology.
- A device that makes use of clothing lights up according to the wearer’s mood has been developed at University of New South Wales, Sydney. It allows the user to communicate non-verbally. Sensors in the garment respond to the wearer’s pulse, muscle tension and proximity, converting physical information into coloured light animations that emulate different emotions like a pounding heart, rush of adrenaline and the feeling of butterflies in the stomach. The device will help minimise injury and provide parents with a convenient, calming and socially-acceptable way of tracking and managing their child’s behaviour.
- A device created at Massachusetts Institute of Technology (MIT) emits radio signals to reflect off a person’s front and back. By measuring heartbeat and breathing, the device can accurately detect the person’s emotional state. Such remote emotion-sensing technology could be used to diagnose or track conditions such as depression and anxiety, as well as for non-invasive health monitoring and diagnosis of heart conditions. The device, named EQ-Radio, has been found to be 87 per cent accurate at detecting whether a person is excited, happy, angry or sad—all without on-body sensors or facial-recognition software.
- Auto-emotive is a project of MIT’s Affective Computing Group, which is based on the exploration of the potential of emotional connection to machines with humans. The goal of the project is to implement mood recognition technology in cars, to create cars that can empathise with the driver and improve the experience of being behind the wheel.
- Development of emotional AI will lead to more effective robotic personal assistants. IBM is developing techniques to add human-like qualities to robotic systems. Such robots will be able to distinguish between, and react to, different people and their emotional states.
- A new sensing technology developed by Panasonic Corp. can detect a person’s emotions in a contactless manner. This information can be used for predicting a driver’s drowsiness and help keep him or her awake.
- Some of the world’s elite coaches, teams and individual athletes have used headsets produced by SenseLabs Inc., USA, which can be connected to an iPhone or iPad via Bluetooth and have dry sensors for assessing brain performance. This sensing technology makes it possible to identify the strengths and weaknesses in problem-solving, multitasking, resource management, decision-making and sleep tendencies. The headsets also provide customised exercise protocols to users to improve their mental facuity, concentration and sleep.
- Bracelets that can measure emotions via electro-dermal activity have been developed and commercialised by many companies. In these devices, the display reflects the user’s heightened emotional states. Q-sensor developed by Affectiva, USA, detects and records physiological signs of stress and excitement by measuring slight electrical changes in conductance and temperature of skin.
Expected benefits
While applications and benefits of emotion-sensing technology are limited only by our imagination, there are some obvious benefits.
- Emotion-sensing technologies can help employees make better decisions, improve their focus and performance in the workplace, manage stress, and also help them adopt healthier and more productive working styles.
- Voice-based emotion analysis in real time opens up many business opportunities by enabling an automated customer service agent to recognise a caller’s emotional state and adapt accordingly. Such information can also be helpful in analysing and managing stress levels of human workers.
- Emotion-sensing technology will also enable companies to establish deeper emotional connections with their consumers through virtual assistants. Data collected through such devices can help companies understand how internal and external environmental factors impact their employees. As a result, companies can redesign processes accordingly, to help keep their personnel better engaged and productive.
- Emotion sensing through wearables can help monitor the user’s state of mind in terms of mental and other health conditions.
Mood sensor technology can help children or the elderly, who are in need of care, to receive timely assistance and support from family members or caregivers, who can then alert doctors.
- Technology that deduces human emotions based on audio-visual cues may enable businesses to detect consumers’ positive and negative moods to better understand their preferences. Such technology can help analyse customer choices that can be utilised for marketing and help detect their annoyances to improve product usability.
- Emotion-sensing smart home devices could provide entertainment (music, videos, TV shows or imagery) to match the user’s current state of mind.
A fridge with a built-in emotion sensor may interpret a person’s mood and suggest a suitable food item for the user to consume.
Similarly, video games can use emotion-based biofeedback technology to adjust game and difficulty levels according to the player’s emotional state.
In the future
Various fields including medicine, advertising, robotics, virtual reality, gaming, education, working conditions and safety, automotive, home appliances and so on will significantly benefit from the use of emotion-sensing technology. Newly-developed smart devices will be able to arrest emotional reactions and convert these into data and facts to analyse situations and come up with appropriate recommendations accordingly.
At present, healthcare and automotive industries are among the most eager to adopt emotion-sensing features. Car manufacturers are exploring the implementation of in-car emotion-detection systems to improve road safety by managing the driver’s drowsiness, irritation and anxiety.
Owing to achievements in computer vision, the emotion-sensing technology field is progressing on understanding human emotions, speech recognition, deep learning and related technologies. Every year, we see new mood sensor technologies being realised. While most of the existing emotion-sensing inventions are based on the use of on-body devices or voice/facial recognition software, research and development efforts are increasingly being directed towards sensing technology that can measure emotions in a contactless manner.
Dr S.S. Verma is professor at Department of Physics, Sant Longowal Institute of Engineering and Technology, Sangrur, Punjab
