System on modules (SOM) and its end-to-end verification using the test automation framework

By Narayan Gour, Softnautics

The purpose of this article is to walk through what SOM (system on module) is, which are all the most commonly available SOMs in the market today, and the benefits of uses. This article will provide a high level overview of how the verification of any SOM or carrier board that we call a SDK, must go through various checks and validations before handing over new solutions to the end user according to their product requirements and how they can contribute to the success of any automated test process.

SOM is a complete CPU architecture built in a small package, the size of a credit card. It is a board-level circuit that integrates a system function and provides the basic components of an embedded processing system – processor cores, communication interfaces and memory blocks on a single module. Designing any SOM-based product is a much faster process than designing the entire system from scratch.

There are several system-on-module manufacturers available in the global market with an equal amount of open source automated test frameworks. If you plan to use System-on-Module (SOM) in your product, the first thing to do is to identify the test automation framework among those available in the market and find a suitable module for your needs.

Image/video intensive industries find it difficult to design and develop custom hardware solutions for explicit application, with reduced time and cost. It is linked to rapidly evolving processors with increasing complexity, forcing product companies to constantly introduce upgraded variants within a short period of time. System-on-module (SOM) ensures reduced development and design risk for any application. SOM is a reusable module embracing maximum hardware/CPU complexity, leaving behind reduced support/motherboard work, thus accelerating time to market.

The system on module is a small PCB board with a processor, RAM, flash, power supply and various I/O (GPIOS, UART, USB, I2C, SPI, etc.). In new-age electronics, SOM is becoming a fairly common part of design, especially in industrial and medical electronics. It reduces design complexity and time to market, which are critical to a product’s success. These system-on-modules run an operating system and are primarily used in applications where Ethernet, file systems, high resolution display, USB, Internet, etc. are required and the application is computationally intensive with less development effort. If you are building a product with a volume below 20-25K, it is viable to use a SOM ready for product development.

Test Automation Frameworks for SOM

A test automation framework is a set of guidelines used to develop test cases. A framework is an amalgamation of tools and practices designed to help QA experts test more effectively. Guidelines involve coding standards, methodologies for managing test data, object repositories, processes for storing test results, or information about accessing external resources. Testing frameworks are an essential part of any successful product release that goes through test automation. Using a framework for automated testing will improve a team’s testing efficiency and accuracy and reduce time and risk.

There are different types of automated testing frameworks, each with its own architecture and advantages/disadvantages. Selecting the right framework is very crucial for testing your SOM application.

Below are some commonly used frameworks.

• Linear Automation Framework
• Module-based testing framework
• Library Architecture Testing Framework
• Data-driven framework
• Keyword based framework
• Hybrid test framework

From the above, modular and hybrid test frameworks are best suited for SOM and verification of their development kit. The ultimate goal of testing is to ensure that the software performs according to user specifications and expectations. The whole process involves a number of test types that are preferred or prioritized over others depending on the nature of the application and the organization. Let’s look at some of the basic tests involved in the end-to-end testing process.

Unit tests : The full software stack is made up of many small components, so instead of directly testing the full software stack, it is best to cover testing at the individual module level first. Here unit tests make sure to have entry/exit test coverage at the module/method level. Unit testing provides a foundation for complex embedded software and delivers quality application code, accelerating the process of integration and ongoing development. Often, unit tests are executed via test automation by developers.

Smoke test: The smoke test is to check whether the deployed software version is stable or not. Further testing depends on the results of the smoke tests. It is also called build verification test which checks if the feature meets its purpose. There’s still development work to be done if SOM doesn’t clear the smoke.

Integrity testing: Proposed changes or features that work as expected are defined by integrity testing. Supposing we fix a problem in the inbuilt product boot flow, then it should be passed to the validation team for integrity testing. Once this test is passed, it should not impact other core functionality. Integrity tests are unscripted and specifically target the area that has undergone a code change.

Regression testing: Each time the program is reviewed/modified, it should be retested to ensure that the changes have not unintentionally “broken” unrelated behavior. This is called regression testing; these tests are usually automated through a test script. Whenever the program/design is tested, it should give a smooth result.

Functional test: Functional tests specify what the system does. It is also known as black box testing because test cases for functional testing are developed without reference to the actual code, i.e. without looking “inside the box”.

Any embedded system has inputs, outputs and implements drivers between them. Black box testing is about which inputs should be acceptable and how they should relate to the outputs. The tester has no knowledge of the internal structure of the module or the source code. Black box tests include sstress test, limit value test and performance test.

Over the past few years, Softnautics has developed complex software around various processor families from Lattice, Xilinx, Intel, Qualcomm, TI, etc., and successfully tested the boards for applications such as vision processing, AI/ML, Multimedia, Industrial IoT and Suite. Softnautics has a market-proven process for developing a verification and validation automation suite without any compromise on feature coverage and/or performance, as well as executing test automation with internal STAF and open-source frameworks. We also provide testing support for future product/solution releases, release management, and product servicing/maintenance.

Read our test automation success stories to learn more about our quality semiconductor engineering services.

Contact us at [email protected] for any questions about your solution or for advice.

Authors biography :

Narayan is associated with Softnautics as a personnel engineer. At Softnautics, he works on multimedia verification, validation and automation testing services for customers. He has more than 7 years of experience in managing product verification based on DSP, ARM platform as well as machine learning and deep learning application development. In his spare time, he enjoys hunting, playing table tennis and listening to music.