Quality assurance teams that work with embedded systems are adopting test automation solutions in record numbers to deal with the immense challenges associated with properly QA testing embedded systems. When it comes to IoT devices, the multitude of platforms and devices working within the environment makes a manual approach to product testing nearly impossible, and the growing number of IoT devices around the world is forcing QA teams to adopt test automation solutions sooner rather than later.
Test automation solutions aren't single product solutions - yet. To run a test on a target device, you'll need a development board with sensors, all connected to a PC, command line interface to flash the test firmware and get readings from the devices, and a testing harness with testing equipment so you can trigger and monitor I/Os, and a test harness. A test harness is a software test automation solution that includes a test execution engine and a test script repository.
As the IoT increases in size, test automation solutions are becoming more sophisticated and giving engineers the flexibility to construct and manage automated tests. Here are the five most significant trends you'll see in test automation solutions in 2019.
Simulated Testing Will Be Required to Verify Safety and Effectiveness of IoT Devices
IoT devices are becoming increasingly common across industries, but one of the most common applications might be in home security. Connected devices such as automatic door locks, smart doorbells, connected cameras and motion sensors all use embedded systems to protect homes from unwanted intruders.
As far as test automation solutions, the simulation will continue to grow in how automated testing is conducted for connected devices in the coming year. Consider the challenges of testing an IoT device used for home security, such as a connected camera. Is it compatible with other IoT security devices that the customer might own? Will they interfere with each other in the customer's home? How will building layouts affect the performance of the device? Given that the product will be used in many different environments, testers will need to use something called service virtualization: modeling different types of building layouts, sensors, and device states to get a better understanding of how devices will function in the real world.
This same reality applies to automated car manufacturers - extensive testing in a variety of simulated environments will be necessary to verify and validate that their products can perform correctly in the real world.
Photo by Agile and DevOps Management Practices Will Grow in Popularity
When it comes to developing test automation solutions, the IoT presents unique challenges for embedded systems engineers. IoT devices have complex testing requirements that combine the characteristics of web and IT services (cloud-based applications with network connectivity) with those of traditional embedded systems, and a diverse set of testing regimens is needed to ensure that testing is done comprehensively.
To promote faster cycles of design, development, and deployment, organizations are adopting agile management strategies as standard practice. This incremental approach to design will affect the way test automation solutions are designed and implemented at the enterprise level. Organizations will conduct software testing more frequently throughout the development process, and each sprint will include the development and automation of test cases that will be used for intermittent regression testing in future development cycles.
Engineers will seek to control and mitigate the development of errors throughout product development by running daily unit and integration tests as part of a Continuous Integration system.
Massive Growth of the IoT Will Make Automated Efficiency a Priority
With the Internet of things expected to grow to over 26 billion devices by 2020, test automation solutions will be required to facilitate product testing for everything from home thermostats to sensors that control traction and anti-lock braking systems in vehicles. Automakers will continue to advance the technology of self-driving cars, making use of complex embedded systems to achieve fully autonomous driving and navigation. Growth for the IoT will also happen in across other industries such as retail, supply chain, and healthcare.
Embedded systems engineers will need test automation solutions that allow them to run more tests with a higher level of frequency and efficiency. As automated testing becomes a necessity for engineers, automation solutions that will enable engineers to run multiple test stations and triage test jobs between them automatically become an essential time and cost saver. The increased volume of testing that embedded engineers do will also lead to the development of more sophisticated test management strategies, including the capability to configure, schedule and conduct automated testing remotely.
Engineers Will Get Better Solutions for WiFi & Connectivity Testing
WiFi testing is a significant challenge for embedded systems engineers, primarily since they typically work in offices, buildings or labs that are set up with WiFi, meaning there is always signal interference. Engineers need to be able to isolate these signals from their device to ensure signal interference does not cause flakiness, and these devices also need to be tested in real-world environments that can differ significantly from the laboratory.
Ultimately, testers need the flexibility to play with all factors when creating test cases for WiFi-enabled devices. That includes the ability to reproduce these kinds of issues from the field in a testing environment to verify that the device responds correctly.
Test automation solutions will adapt to this need by offering emulated networks that give engineers the flexibility to better test WiFi connectivity for their devices. Emulated networks will include features such as injected packet loss, throttling, and other mechanisms that can replicate the real-world environment. Leading manufacturers and developers of IoT devices will also invest in physical products like RF isolation boxes, which can be used to shield a device from network interference during functional testing.
Test Automation Solutions May Develop Compliance Approach
Test automation is a high stakes proposition for the major car manufacturers that work toward perfecting automated vehicles with embedded systems. When these organizations conduct software testing, they'll be validating the safety and effectiveness of vehicles that will be put on roads with no driver and expected to function flawlessly. Car manufacturers are aware of major potential hurdles with insurance or liability they may face if autonomous vehicles are involved in collisions.
As of now, the legal status of self-driving cars is up in the air. States like California have passed guidelines for testing autonomous vehicles with drivers present in them, while New Mexico and Montana have no laws at all for driver-less vehicles. With road safety and privacy considered severe risks, providers of test automation solutions should anticipate that future regulatory oversight of self-driving cars could include things like a quality management system for algorithm and software testing. A recent article in Scientific American proposed that regulators should test algorithms/software directly, the same way they require testing for mechanical parts and electronics.
As recently as 2016, the federal government required automakers to submit a 15 point safety assessment to regulators. As more collisions involving automated vehicles are reported, we may find the federal government moving to regulate the quality processes, including software testing, that underlies technologies in the automotive and even other industries.
There could be a competitive advantage for auto-makers who can validate their test automation procedures and verify the safety of their products more quickly, reducing time to market. This may not materialize in 2019, but it's a change to watch for over the next five years.
Summary
Overall, test automation is a growing area of interest for embedded systems engineers that will help improve the safety and effectiveness of IoT devices while reducing their time to market. In 2019, more organizations will adopt Agile and DevOps management practices and increase their efficiency and decrease time to market. Test automation solutions providers will focus on features that make it easier and more efficient for engineers to conduct testing, and simulated environments will increasingly be used to verify the effectiveness of devices in real-world conditions. Engineers will adopt solutions to enable easier WiFi and connectivity testing, and the government may play an increasing role in regulating the way that engineers test devices that could pose a risk to the public.
Stay ahead of the game and learn more about how these embedded systems solutions can work for you.