Incidents like what happened to the Toyota Prius makes us take a second look at how secure the code running in our cars are. With the IoT paradigm steadily making its way from fabs into embedded systems in every vertical, seeing them in the automotive sector is not a surprise. How can engineers ensure safety in a car that has enough code to virtually have a mind of it’s own?
Shinto Joseph, operations and sales director, LDRA Technology Pvt Ltd, speaks with Dilin Anand of EFY about the changing face of embedded software testing in the automotive sector.
Q. What is the biggest design challenge faced here while implementing safety solutions?
A. The challenge is beating the memory limitations. When you look at a car manufacturer, every penny counts for them. When someone suddenly asks a car manufacturer to follow all new international safety standards, they will have to innovate and reuse whatever is available. Why? Be
cause if you plan to redo everything there is going to be millions of dollars in cost overruns. As far as the vehicle manufacturer is concerned, he has to work within these limitations with the available limited memory. Ideally when we are testing these sorts of car systems, the code will become bulged because of the instrumentation that we do within the code. As we begin writing code for instrumentation, the original code will be bulged by anywhere between 20 – 30 per cent depending upon the nature of the code. Eventually the code will not to sit into existing hardware. Now, this is where the car manufacturers will start scratching their head.
Q. Interesting. So what’s the solution for this issue?
A. Well, we have faced similar problems in aircrafts before. In the olden days, the kilobytes (KBs) used to matter because the memory was packaged in electronically programmable read-only memory (EPROM). Today’s universal serial bus (USB) and similar standards were not available those days. The test cases also used to account for the KB based scenario. Today, the semiconductor-based concepts came and brought solid-state electronic devices with them, enabling us to do all these tests within their limited memory requirements.
Q. Could you give us an example of a technique followed while doing this?
A. The engineer could for instance, could tackle the testing partially instead of doing fully at one go. Engineers can look for the limits at which they are operating — that means if you have a constraint of 30 per cent, then we need to make sure that at any point of time the code you are building should not exceed the upper limit (30 per cent in this case). In some cases you really need to know the limitations of the hardware and do partial instrumentation tests, which eventually should fall within the international safety guidelines else the car will not get a pass through the safety- security tests. This is known as limited memory testing.
Q. How good are the automotive standards for which manufacturers are going through all this trouble for?
A. While current solutions of the automotive industry have not thought in this direction mentioned above, the sector does have a new standard ISO 26262, which is actually a subset of the aerospace safety standard. In the olden days the enforcement of these standards were limited to aerospace and nuclear. Later, other domains like railways started using it because anything involving public life got people worried. Then you had medical devices following this with industrial things like lifts, elevators and automotive. We are now reaching a level where the forecast is that cars will have more code running than aircraft — a scenario with the IoT-enabled car.
Q. What do we need further tests on cars like this, if the code runs well within the design parameters set for it?
A. Let us take a scenario, where I want to see how a code is running inside an embedded device. The code might run well within the design parameters in the host system. However, the behaviour could be slightly different in a target device due to various reasons. When you are using it in the target system like a car, you have actual environmental factors coming into the picture. You might have seen automatic wipers in operation. There are external factors that are coming in, and some times you will come across a one-in-a-lakh situation. Cases like where you are running a car at a very high speed and at the same time you are seeing an iced patch on the road. It is a very rare combination, but you can’t say it would not happen. It may be a one-in-a-million combination but there is still that one chance. The car should be tested for that scenario. The only way you can predict such a situation is by doing a thorough test on the actual target situation using instrumentation.
Q. What is instrumentation in your context, as I am more familiar with the conventional T&M using probes and scopes?
A. We take a copy of the code, and do something called instrumentation on it. What we do here is that we add some more lines of code between the existing lines. These additional lines help us to see how code is running in an embedded device. This code is added based on study on the nature of the originally existing code, how it behaves, and how it could behave. Then we run the added code in this particular environment itself and dump that instrumented code into the target. What happens here is that the code size also goes up. That’s where the issues start. Now if I want to see the behaviour of what is happening inside – we call in history analysis. I would have done one run in the target and then I want to see what is happening during this run. I can simply take back the history file; bring it into the test environment. Again some one can go through that to find out what was really happening.
Q. What happens when the car being tested is an IoT-enabled automotive?
A. If you want to do a thorough test on the target with so-called ‘instrumentation’, you will need to study the behaviour of code and target. In a real world scenario this is very difficult because you will have a lot of sensors coming in, with each sensor converting the electronic angle parameter into the digital signal, which in turn becomes an input into the actual hardware. Now when something like the Internet of Things comes in to automotive, you will have hundreds of these sensors that are in turn controlling an artificial intelligence system.
Q. There must be a solution to test such a system, could you share some insight on how you would do it?
A. In such a case, I want to slice them into various instances and then do minute analysis to see what is happening. For this I use something called reverse designing, which is done based on the trace data. Here you run the system and keep monitoring the data that you glean from the output. The output from the board is used to trace data on the microprocessor, which will be in binary, and this is then placed into another area of the system. So even if the system is crashing while running, you still won’t lose the data. In most cases when the system crashes, you won’t even know why the system crashed. This technique allows you to pull out the trace data and put into a safe area so that the data is always available. It works almost similar to a black box.
Instrumentation is a different form that is used for testing, while trace data analysis is a developmental technique. These give two different angles to look at the system so that even if the design could not find it, at least the tester is able to find it.
Q. What’s are the problems a company can face if the design team doesn’t take care to ensure the integrity of the code running on critical systems?
A. There are no issues until there is a safety problem that has caused an accident. Once that happens, there are no issues until there is a call back. If there is call back that has a safety implication an investigation into it is going to be mandatory. During this investigation, if it is found that the manufacturer has produced a safety defect because he didn’t follow safety standards, then he is liable from a criminal point of view, and from insurance too. The party who is not conforming to standards, or those who did not have the artefacts interpreted by a certification agency like Underwriters Laboratories (UL) and Technischer Überwachungsverein (TUV) will be in trouble. If the above are conforming and done, then they are only liable for the accident damages to a limited amount. Else, the company will be totally liable and the senior management can end up behind bars as well, while the company itself gets sued for millions of dollars.
Quick question: Is it safer to buy a car tested in the real world or one that has passes the ISO standard requirements? Can we completely rely on software?
A. Technically if you are asking me whether an insurance cover is better than previous car’s software, I still don’t know. To be frank, if I take a car that is running for 15 years, I assume the car is safe. That is why people are still using it around the globe. On the other hand, the new standard is good. It is based on the aerospace safety standard, which is the safest transport today available in the world. And since it is a subset of and more or less the same as the aerospace safety standard, people expect it to reduce the recalls, while at the same time making more and more people accountable for the money they are burning into the car.
Q. Is there any difference between the standards followed within and outside India?
A. In any country including India, there will be local regulations for product safety. An apex regulatory body will do this, which is the Bureau of Indian Standards (BIS) for us in India. Normally people associate gold with BIS, but they actually do much beyond that. Every industry should listen to the BIS before they release any product – this could be with respect to safety, security, or power efficiency.