Automated Testing for APIs

API testing with SoMA is more than a traditional API testing. Normally, the API testing concentrates on verifying the compatibility of the API methods (or functions) and testing the parameter ranges of each method. As the following table describes, API testing of SoMA verifies also the dependencies between methods and also verifies the robustness by testing illegal method scenarios.


Three steps of API testing:

1. Parameter testing: Compatibility verification of the methods by testing the smallest and largest input parameter values, as well as random values between them. Furthermore, negative testing is done by providing illegal input parameter values to the method interface and verifying that the correct error code is returned.

2. Correlation between inputs: The API testing verifies the correlation between methods. Normally, there are two kinds of dependencies between methods: callbacks and set-get method pairs. Testing verifies that correct GET values are returned for each SET input. Also the correct response of illegal input values is verified. In addition, the callback logic also is checked by detecting the response (callback) to each call.

3. Method scenarios: The API testing verifies the method sequences and the stability of the system when illegal method sequences are executed. This has revealed to be a very effective and stressful test to the camera systems and several robustness improvements are done after the sequence tests.

The API testing also offers the possibility to compare the API compatibility and robustness between different devices. When the existing API tests were executed against commercial devices, the pass rates were as low as 57% and severe crash issues were identified.

The API testing can be implemented against every API in several platforms. Currently, the SoMA API testing has large reference implementation against Android CameraHardwareInterface. The API testing contains about 200 different tests against this interface. CameraHardwareInterface is the device driver interface of Android camera system and it offers C++ API. However, the same kind of tests can be done, for example, against Java-API.

SoMA also has the option to execute external tests, such as, Android Compatibility Test Suite (CTS) tests. When SoMA API testing verifies the device driver interface, CTS ensures the compatibility of the application layer. Together these features create a very comprehensive testing entity of the whole camera system.