Traditionally, the left side of the V-diagram has been viewed as an R&D or design task and the right as test and validation tasks, and these two phases of product development have been separated. Once the production control unit is manufactured, it is passed to the validation engineers with a set of requirements to complete the verification & validation (V&V) process, and then report back on the results. However, in the concurrent design and test approach, there are real-time testing tasks going on throughout the entire process. “In the design phase, engineers perform model-in-the-loop testing by applying stimulus profiles or test scripts to simulations of the system and analysing its response. Once an algorithm is ready for prototyping, engineers take the prototype controller to the test lab or field test site for evaluation after which it is deployed. Testing occurs throughout the process,” adds Chivukula.
T. Anand of Knewron also stresses on the importance of testing during design, and cites an example where it helped his firm to remedy a problem quite inexpensively. “In one MCU-based design, we found that RTC value was not synchronised; rather it was giving random timing whenever invoked. Majority of the system functionality was dependent on RTC sync and so was a critical issue. By logic everything should have worked fine, but it didn’t. We could get out from that issue after changing declarations of RTC-related variables and also changing their types. Additionally, we had to change compiler/linker settings where it was optimising unnecessarily and causing trouble. If we would have skipped that test (assuming that RTC code is common and can’t have any issue) we would have spent too much time in final phases during release testing, which is costlier than early detection and remedy.”
Advantages to the design process
As mentioned earlier, in many cases, it is not possible to complete a design without testing during the design phase—especially for new technology. Apart from that, the other major advantages are risk reduction and cutting costs. If a design has a problem that is only discovered after the prototype is almost complete, the costs of correcting it can be enormous. Stephen Hire explains, “If the problem is at the heart of the design, then a lot of rework will be required; and where companies are pushing to meet a specific launch date, a problem found late in the design phase can cause big delays. Or even worse, the problems may only be found after the launch when customers start complaining and returning products to the manufacturer—every manufacturer and designer needs to minimise the risk of this happening.”
Teledyne Lecroy spokesman gives us a different point of view, “Product development is fundamentally a risk mitigation exercise, and testing gives you confidence that the risks have been mitigated. If a given subsystem has been thoroughly tested and verified, it should not be the source of issues upon assembly and testing of the entire system. If all of the subsystems have been similarly verified, faults in the end product can typically be traced to issues with the interfaces between the subsystems. Particularly with larger systems, testing during the design phase helps to partition the problems to achieve faster time to market.”
In the design of any large, complex system, testing during design implies that you know where you are in the process. Knowing where you are as you design can inform what you do at the next step of the design process. In general, if implemented properly, testing during design reduces, and has the potential for eliminating, long loops of redesigning. As a result, the development process tends to be more linear and predictable and, consequently, the time to market is reduced.
In short, the advantages can be summarised as: (a) an ability to test leading-edge functionality where there is no alternative; (b) enhanced ability to optimise the design for superior performance; (c) a lower risk of design problems being discovered late, causing costly rework or delays; and (d) reduced risk of financial losses or reputational damage.
Challenges
Most leading consumer and industrial electronics manufacturers already use this methodology, because they understand that the costs and risks of not doing so are too high. Also, if they want to be ahead of the competition, they cannot wait for technology to mature—so they need to test during design. However, for companies working on mature technologies, some regard the costs of investing in additional test equipment early in the design stage as too high. Stephen Hire agrees, “It is easy to quantify the costs of extra test equipment but many engineers forget to factor in the costs of product launch delays, excessive customer returns or damage to a company’s reputation from inadequately tested products. Let’s take an example of a product with a selling price of Rs 5000 and an expected monthly sales volume of 10,000 units. Every one week of delay for the product launch will cost the company approximately Rs 12,500,000 in lost sales revenue; and that doesn’t even account for the cost of faulty products returned or the damage to reputation. A shortcut in the design stage can quickly add up to a huge financial damage.”
A typical challenge of testing during design is of simulating the conditions of the overall system during lab conditions. “In order to simulate the right conditions, it typically calls for extra effort in writing test cases and test programs, which some people might consider as an overhead during design. And because of this, some engineers might refrain from doing so,” adds Amit Sinha.
In general, the time-to-market pressure is the biggest factor. There is also a lot of pressure to see the entire system working. Unenlightened management might see an involved process of thoroughly testing subsystems as a waste of time. According to Teledyne Lecroy spokesman, “There is often a conflict between testing at the system and subsystem levels, and achieving a balance between them takes much thought and planning. Designers who are trained primarily in design alone must think about the testing as much as the design. More experienced engineers do a better job of factoring in testing, because they have suffered the consequences of ignoring it. Engineering schools do not teach their students to approach design with testing in mind. Rather, it is learnt with time and experience.”
