New-generation embedded test tools enable software developers to be confident of delivering high-quality software, making a significant contribution to new products that work first time
Magnus Unemyr and Mark Moran
Releasing a product with bugs is potentially very expensive, especially considering the cost of field ugrades, recalls and repairs. Less quantifiable, but equally important is the damage done to the reputation and consequent loss of customer-goodwill by any company. Notwithstanding, many embedded products are indeed released without all the necessary and desirable testing. Consequently, software quality is receiving increasing focus in embedded application development.
Most existing software testing tools can only execute the tests on a PC, the use of which is limited when testing an embedded application. The actual system has numerous differences including hardware interfaces, timing issues and memory constraints among others. Therefore it is important to run as many software tests as possible on the actual embedded hardware to minimise inter-platform differences.
The challenge is to findautomated in-target software testing tools that can be easily integrated into developers’ embedded development environments. These tools should be able to analyse the application source code, autogenerate a test suite and automatically run it on the target board. However, a test report showing 100 per cent success may not be sufficientas the test procedure itself may not have been applied to more than a small fraction of the code. Understanding the quality of test procedures thus becomes critical in judging whether a product is ready for release.
However, new and advanced tools for embedded test automation and test quality measurement are now emerging.
Unit testing
A key element of the software testing process is ‘unit testing.’ Unit tests call up ‘C’ programing functions with different combinations of input parameter values to drive the code through different execution paths. Other than for trivial functions, writing unit tests by hand is labour-intensive, costly, tedious, and might leave many possible execution paths untested. A detailed analysis of how input parameters drive the code is therefore required, so that important execution paths throughout a function are actually exercised by a test suite.
Another challenge faced by development teams is maintaining the synchronisation of unit tests with code under development. This problem is often exacerbated when delivery schedules become compressed with project progression. If source code and unit tests become out-of-sync, the tests will become less useful, especially when these are needed the most.
The unit test tools available to PC developers are less useful to embedded developers as these rarely manage compilation, downloading and execution of the test suites on embedded boards. Tools that create the unit tests automatically, build these into the embedded application and allow to run on the target via a convenient pathway, such as a joint test action group (JTAG) debugger, result in optimal usage of time and maximum productivity.
An effective approach is to use a fully-embedded test automation system that is integrated right into the C/C++ integrated development environment. Thus synchronisation is easy to maintain and ancillary tools such as JTAG debug probes can be used to full effect. Such tools were not common previously, but new tools now meet these demanding criteria and bring powerful in-target test automation capabilities to software developers.
Unit test automatic generation. The new breed of professional embedded test automation tools can analyse the source code and generate and execute suitable test suites automatically on the target board. For example, these auto-generate a test suite and make many function calls with different combinations of input parameters, thus driving a large number of different execution paths.
Unit test automatic execution. Once the test suite has been generated, it must be compiled, linked and executed on the target system. Many low-cost unit test tools exist that run unit tests only on a Windows PC. The inability of these tools to integrate with the embedded tool development environment results in additional inefficiency to execute the tests on the target board. However, the new breed of professional embedded test automation tools integrate into a professional embedded IDE, enabling easy synchronisation of test with code development. These also take advantage of integration with tools such as JTAG debug probe.
Measuring test quality
Once a code has been developed and tested, the focus shifts to quantitatively understanding what actually happened during testing. Code-coverage measurement, which is performed using dynamic execution flow analysis, i commonly used to study which parts of the code have been tested.
