Although not necessary for all applications, there is a growing demand for technologies like gesture control, to enhance user experience, or to solve challenges presented by external factors like the environment. For some users, interaction with the two-dimensional screen of a phone may just seem too flat, even with the haptic feedback that most contemporary phones offer. Other users will struggle with the fact that touch technology requires physical contact – which just might not be possible in some environmental conditions.
Touchless gesture control uses sensors to capture a range of motions, adding z-axis sensing in the free space around the sensor. When results are compared with others in a software library to interpret the action, instructions can be sent to the system controller for processing. For users in harsh environments, unable to perform physical contact, it can provide immediate benefits, for example, enabling operators to undertake actions whilst wearing gloves or from behind a physical barrier, so that the unit can be completely enclosed, providing further protection from the elements.
There are many touchless gesture control technologies that are popular in the market today. Console peripherals, such as Microsoft’s Kinect and Nintendo Wii controllers allow users to control actions on a television screen, and more recently, Samsung has led the way in the consumer electronics market by allowing users to control its televisions using gestures from across the room. Touchless gesture recognition even made its way to the automotive sector when BMW incorporated the technology into the 7-series, to deliver a variety of simple control functions.
The two most popular sensors used in touchless gesture control applications are cameras and electric-field (e-field) sensors. Camera sensors are usually found in complex, higher-end applications, like the Microsoft Kinect. E-field sensors are simpler and less expensive, making the technology ideal for a much wider range of applications and can also be placed behind non-conductive materials and operate normally.
Design Considerations for Touchless Control Applications
Being a new technology, touchless control applications have new design considerations. Touchless sensing does not offer the luxury of being able to provide prompts and feedback via a screen itself, which can make the process easier, so designers must make firm choices at the start of the design process to compensate. Priority must be given to making the experience feel comfortable and familiar to users. It can be helpful to map out a model of the system and all the potential options required before setting to work on physical design. Getting a clear understanding of all the options required and associated gestures enables the designer to identify the type of information a user might need to navigate the system. Thought should also be given to how a sensor’s location be identified if it is behind a barrier & the type of gestures to be used and how a successful reading can be confirmed.
Most designers include a screen that can convey information and provide feedback to the user, however, some other techniques that have been used for touch control are also helpful for touchless applications. When developing for the screen, signifiers are important to the context of the application. Often two different programmes will use the same gestures for different functions. The contextual nature of the gesture should be reflected on-screen to make operations easier and more natural.
Even the natural environment where the system is placed is an important consideration when designing a touchless system. Visual cues play an important role when a user operates a piece of equipment for the first time and these cues, called affordances, can help the user become familiar with the system. The system should be designed to feel natural to the user – for example – it is normal for humans to receive a response when a gesture is made, and if no response is received we can become confused. Mobile phones designers know this and often use haptic feedback to let users know an input is valid and accepted. Feedback is even more important for touchless gesture controllers as there is no physical contact. For example, it prevents errors that can come from multiple repeated gestures, which can happen if a user is unsure if an input has been accepted.
Practical examples
Microchip is one company that provides a complete ecosystem for designers looking to build touchless gesture control applications. The products go under the company’s GestIC banner and are built around its MGC3X30 family of gesture controllers and Aurea GUI software.
The MGC3X30 gesture control chip offloads the gesture recognition functions, leaving the main system controller free of overheads. The low-power products offer a detection range up to 20cm and contain all the building blocks required to develop a single-chip input sensing system. To provide designers with an easy way to evaluate the technology, Microchip has also developed a variety of development boards. element14 currently has the Hilsar single-zone development kit and the Sabre Wing dual-zone board available from stock.
There are several hardware options available for designers; one example is the ADI ADUX1020-EVAL-SDP, a gesture and proximity sensor evaluation board. The kit provides users with a simple means of interfacing with the sensor (ADUX1020), collecting data from it and evaluating gesture recognition capabilities. It requires an evaluation tool that can be downloaded from ADI. This is a graphical user interface (GUI) that provides low and high-level configurability, real-time data analysis and user data graph protocol (UDP) transfer capability so that the evaluation board can easily interface to a PC.
Another hardware option for designers is Flick HAT, for the Raspberry Pi. The add-on board uses the Microchip GestIC technology to allow designers that have a Raspberry Pi or compatible board to have easy access to a powerful gesture control system. Flick HAT can plug straight in to Raspberry Pi variants, Pi A+, B+, 2B and 3B. It allows designers to control devices by using familiar gestures, which can be up to 10 cm from the sensor board. There are many code examples available that can be downloaded from github.
Different versions of the Flick HAT are available for other development boards, for example, Flick Large is compatible with Raspberry Pi, Arduino, BeagleBone and Genuino, and any other I2C-enabled device. The Raspberry Pi Zero is supported by the Flick Zero.
Summary
Touchless gesture control is an exciting technology that can complement existing touch technology or be used to replace it completely. The technology opens new applications and new ways to naturally interact with machinery. Although there are some differences in development when compared to touch-based applications, there are also many similarities, primarily the psychological techniques that help humans easily gain familiarity with the technology and make it easy and natural to use. Touchless control technology is easily accessible, either through a tailored ecosphere or through an add-on for popular development boards.
