This marvelous innovation with great features and benefits is not just a boon for novice DIY programmers, but also an example of AI put to the right use.
It is quite fascinating to come up with an electronics DIY project idea and envision its benefits for solving a simple yet compelling problem. But when it comes to building it, especially doing the coding, many DIYers face numerous challenges with respect to learning a programming language and effectively debug the code.
In order to resolve this long-standing issue, researchers from KAIST, South Korea, have developed a conversational tool called HeyTeddy that makes it possible to do programming without writing code – in real-time.
Voice commands
As seen from the image below, HeyTeddy’s voice input is handled by an Amazon Echo Dot, which passes the voice commands through the cloud to a Raspberry Pi. The system then interacts with the physical hardware on a breadboard using an Arduino Uno that is running the Firmata firmware. An 18cm (7-inch) 1024 x 600 LCD touchscreen has been additionally placed for the GUI. Once programmed, the code can be exported and used on the board by itself.
Commands that are spoken to HeyTeddy undergo through parsing for them to be executed. For example, the “write high” command configures an I/O pin to behave as a digital output with its internal state set to high (e.g., a 5V logic level). By simply saying the command, the HeyTeddy system automatically writes it into the code. This saves the user of going through the trouble of writing the code, then compiling it, debugging it and finally, manually uploading it on the hardware.
Which one is better? HeyTeddy or Arduino…
To check the efficiency of conversation-based programming as compared to traditional programming, researchers tested the HeyTeddy system on three parameters: obstacles, breakdown and bugs. All these collectively define the problem generally encountered during the prototyping phase.
Each of these refers to programming barriers, programming disruption and programming faults.
After a comparative analysis between voice-enabled and text-enabled programming, it was concluded that HeyTeddy outperformed Arduino by being 26.1 percent more faster. This led to a higher completion rate and fewer obstacles, breakdowns and bugs.
In traditional programming, users typically typed the code in the Arduino IDE and manually uploaded it on the Arduino micro-controller. But for conversational programming, they instead had to speak out the commands to HeyTeddy, which saved a lot of time that would have been spent searching for command and syntax details.
Preventing learning barriers and bugs
Besides offering a feature-rich set of custom Arduino components (without writing any software), HeyTeddy can efficiently handle ambiguous requests by understanding the context (of the code) or ask the user to clarify further, in case of doubt. Generally, a compiler doesn’t issue a warning if an error is committed while writing the program. Such as, configuring a wrong I/O pin. However, HeyTeddy promptly alerts a user of any mismatch or changes the state of the pin so that it is compatible with the command. This way, any learning barrier or internal bugs are prevented.
Many Arduino users first build the entire hardware and write code. But the conversational programming with the help of HeyTeddy allows users to work with both aspects parallelly. This enables better debugging practices to solve errors while testing each component without carrying over to the next step.
Limitations
The limitations of HeyTeddy are related to the complexity of the prototypes, the level of customisation of the workflow and the technical flexibility of the hardware and software features. Complex circuits might be harder to debug and the system currently supports only one user at a time.
Further improvements
As of now, there is no immediate plan for supporting multiple users; research is going on in making multiple voice input for multiple users a possibility. The current system supports 12 customisable components with plans to extend that number. Based on the user expertise level, several Arduino language commands will be accessible.
The current system lacks an “edit” feature that allows users to modify previously issued commands. Researchers are also aware of this and focussed on including this absent feature possible for users to provide complex commands and programming constructs. and will further demonstrate the practical benefits of using a conversational agent for physical computing prototyping.
All the above indicate that voice input is preferably more suitable over text programming with simple logic. Voice encourages explorations and reduces the cognitive load that is needed to focus on two different tasks (programming on a computer and assembling hardware on a breadboard.
Read the original research paper here
Thanks for the information, we all know that the we are stepping up to the world of AI s and voice searches. I think this is one good blog I have read recently.