Designs and Projects Development designs and develops products a variety of products focusing on haptics and mobile robotics. They have haptics based products targeting surgery simulation, mission control, and assembly operations. The products designed here, are then sold though another concern, Merlyn D3 Systems.
Mr. Biju Ronnie Varkey, Owner, Designs and Projects Development, talks to Abhishek Mutha of EFY, about haptics, and challenges faced while designing advanced bomb-disposal bots.
Q. Could you start off by giving us an overview of your organisation?
A. Designs and Projects Development designs and develops our products, and then we sell this product under the name of Merlyn D3 Systems. Merlyn D3 Systems can be called an innovative arm of Designs and Projects Development, and is a company where we market our range of mobile robotic devices, which can be plugged together to form any type of device.
Q. Currently, what is your R&D team working on?
A. Right now our main focus is developing sub-systems for haptic devices for which we have our own mechanical tool room, electronic development center, and software development lab. We are also coming up with other technologies as well. The two focus points for us are haptics and mobile robotics. Mobile robotics is a device for harsh terrain which can move in any direction – it’s used for bomb disposal. Haptics technology is being used for surgery simulation, mission control, assembly operations, and many more areas.
Q. How would you explain haptics to a newbie? Why is it gaining importance?
A. Today we are in a phase where haptics is necessary. For example when an arm in a robotic device is attached with a tool like a drill, and a suitcase with an explosive device is being drilled, then the amount of force required to drill through it is very important for a very safe disposal. Otherwise, the damage caused can be very huge. The force applied by the robot has to be felt by the operator otherwise there can be possibility of tremendous damage. If there is a haptic feedback, the operator knows exactly how the arm and the tool are operating and also the forces that are involved.
Secondly when a trigger is being removed from an explosive device, it’s important to know the pressure being applied by the gripper. Too much force might cause an explosion and too less will make it impossible to extricate the trigger. There should be a very precise control where in the operator should feel that he is at the site performing all the operations. We are also using this technology in surgery simulation also.
Q. How does haptics work?
A. Take the simple example of riding a bicycle. We sense so many factors including the position of our body, the speed, the rumble of the road, the wind on our face, the exhilaration of gliding, the force we apply on the pedal and the effect it has on acceleration. Try doing that on a game console on your PC. You now know what you are missing.
According to medical science there are more than 28 different types of sensors in our human body. These sensations can be used as a feedback mechanism to add to our feel of our action as to its implementation by the machine being controlled by us.
Haptics is the science of feel for our perception senses so that our body can experience the result of our actions to control a robotic system. In a haptic system the end effector is attached to sensors to detect the process being undertaken. The force and orientation data is fed to the controlling device and the action is mimicked so that the operator can feel the effect of the actions being given through controlling device.
Traditionally the feedback was limited to the sense of force that is being applied by the end effector on our robotic system. So the force that is being applied is fed back to another robot actuator system that is integrated to our control device like a joystick. The feedback would be in a ratio as decided. It can be amplified if the force applied is very low so that the applied force perception is detectable. It can be as a lower ratio if the forces being applied by the robot are high.
The above definition of haptics will also allow a camera and display system attached to a robot to be called a haptic device as the operator guides the robot while viewing his actions. This form of haptics would be called a visual haptic system as the visual perception is used to control the robot system.
Q. How is the technology advancing?
A. As technology is advancing more types of human perception are being used to function as a control feedback methodology to actively guide the robotic system. Sometime they are also being used as combination to achieve more complete involvement.
Below is a view of the different ways the haptic device can be implemented. The haptic device can have a feedback which is localised as in the vibration of a mobile handset. It just intimates that your attention is needed and you have to look for it. It does not have any direction to point you in any particular direction for your attention. Some devices can apply forces in a particular direction or vector. That is they can apply force in a 3D space of a particular size. These can be used to guide robotic arms in 3D space while knowing what are the forces being applied by the end effector. Sometimes more than one type of sensory input may be used to increase the feel factor which we call the multimodal interaction method. The haptic information can be used as the primary source of information to control your system or can be used as secondary information channel. Secondary signals may be like warning signals that a potential dangerous situation is being approached. The warning tone on your PC when you do some wrong command with your mouse is a prime example of secondary haptic audio information being provided by your ears.
Q. What are the key components in an haptic-based system?
A. The key components would be actuators, sensors, effectors, controller and drive circuits, it’s software and displays for the user interface. Different types of actuation mechanism can be used to stimulate different types of sensory inputs to the human body. For example electrostatic technology can be used to stimulate friction and normal indentation. Piezoelectric technology can be used to stimulate normal indentation, lateral skin stretch and vibration. Similarly a particular feel can be stimulated by different technologies. For example normal indentation can be stimulated by electrostatic, piezoelectric, motor, shape memory alloys, electromagnetic micro coils, air jets, pneumatic valves and dimples, acoustic radiation pressure and electro stimulation.
Q. What are the main design parameters for haptic effect?
A. The main design parameters for haptic effect are:
1. Duration: How long the haptic effect will last
2. Envelope: It is the area that bounds the attack and fade portions of an effect (how the effect starts and ends) and allows you to change the level of the effect over its duration.
3. Magnitude: The strength of the output of the sensory perception actuator
4. Period: The timing between the sensation pulses.
5. Phase: It sets the point at which the effect begins with respect to an event.
6. Wave form: The shape of the repeated sensation pulses.
Q. What are the key design challenges in designing a haptic-based system?
A. The main challenge in designing the haptic device is to make the control interface to feel exactly like the tool being used originally by the operator. Take the case of a haptic device that mimics the tools of the Endoscope Surgeon. The haptic device has to feel like a normal endoscope. The actuators and sensors have to be implemented to look almost like that. The major issue would be balancing the weight of the different actuators etc. along the different axis so that the surgeon feels exactly like his original equipment. Another design aspect would be to provide the kinetic motion and freedom exactly like he would feel while conducting the said operation on an actual patient. It is also very important to make the software and control mechanism fast and accurate.
Q. Could you talk about an interesting technical challenge that your design team faced recently?
A. When we were designing the haptic device for surgical simulation, one of the major challenges was balancing the weight of all the mechanical components and nullifying the imbalances of the weight. When an actuator is used, it adds weight to the device. So we had to design the structure in such a way that they all cancel out. The second part of it was that we wanted to give it a natural motion path to the operation of the device. If endoscopic operation is considered, we have to actually follow a path that a doctor in actual situation will take. So the whole device has to be made as if the doctor feels that he is actually operating. We had to juggle the mechanical designs and the software feedback in such a way that we had to actively overcome the weight with the software. It was challenging but one fine day the idea clicked that the whole thing can be done. We came up with a device which had 5 degrees of motion out of which only 1 degree of motion had to be actively stabilised whereas the remaining 4 degree were naturally stabilised as far as the weight and friction were concerned.
Q. What are the new innovative applications emerging for Haptic-based systems?
A. Haptics can be used anywhere depending on your imagination. When there is an operator involved in commanding a robotic system where application of forces is critical, a haptic device becomes a must. In a normal system, joysticks are used and the only thing known is the extent of the motion not the amount of force. We are using haptics for mobile robotic system for bomb disposal and surgical simulation, where the doctor can ‘practice’ his operation that he is supposed to do on a live patient or try to simulate different ways in which he can perform the same operation from different locations. We are trying to use haptic devices for assembly simulation. Most of the haptic devices that we have developed are passive devices or active devices where the device pushes an operation being performed like motor actuators.
Take the case of vehicular route mapping devices. Presently the driver has to look at the device to know where to turn. Now if the device is having a directional vibrating mode where the direction and amplitude of the vibration is dependent on the nearness of the turn. The driver can place the device in his pocket. As the turn approaches the directional vibration will tell the driver about the turn coming up and the direction. He does not have to take his eyes away from the road and it will be much safer for him and others on the road.
Another main application area in controlling an exoskeleton strapped to the human body. Especially for a soldier it will be easier for him to carry larger loads and control heavier machines. Haptics are also being used to mimic terrain realism. Consider giving a real feel to the simulation of different terrains a soldier will encounter while running over a rough mountainous terrain. The reality will add to the training efficiency of the training methodology.
Haptics is being used extensively used to teach surgery and stimulate different surgical procedure. We can train our doctors better and with lessor cost.
It will be much better in simulating machine control. Take the case of flying instruction or operating heavy machinery or even for remote control of heavy machinery. The list is endless and just depends on our imagination. Haptics can be very usefully be integrated into our education system to teach students about how to assemble different equipment. Try teaching 100 diploma students how to assemble a laser printer year after year. Try doing this every year. Just look at how many laser printers will be destroyed. Whit this technology you can teach the same student to assemble as many different model of equipment as we can imagine. We will be giving practical and effective education and will be producing employable workers.
Q. In which application vertical do you see great demand?
A. In the coming years, according to me, there is going to be a very big scope of haptic devices. We can compare it to how a mouse changed the operation of a computer. In the coming years, Haptics is going to change fundamentally the way we use our computer systems. Presently in India, I see that most of the work that we try to get out of the computer is output through the printer or the display. But once this technology comes down, it will become much more easier for us to control machines where you can do things which are not possible right now. Haptics is going to be a major vertical in future considering the demand.
Q. Basically what kind of audience do you target?
A. As far as haptics is concerned, everybody is a potential user. It has not been elaborated to people as to what this technology can do. We try to educate as many people as possible in every field. It can be used medicine, construction and many other fields. In this field it’s very difficult to judge where your next potential customer is going to come from. Some from woodworking could also be the next potential customer.