Performance testing reveals bottlenecks in the camera system by measuring the execution times of different camera functions and combinations of functions. Very high resolution camera sensors can cause critical performance issues when megabyte sized images are processed in a very short time.
Performance testing creates timestamps when the camera API calls are made and when the corresponding response is received or callback method is executed. The execution time is simply the difference between these timestamp values.
SoMA also measures and records the execution time of each camera function. For example, the compression time of the encoder can be measured as can the shutter speed with different auto focus settings. Such measurements are quite straightforward however, different combinations of features may change the performance values significantly. With SoMA it is possible to measure different combinations and identify the critical ones.
In general, performance tests do not require reference images. However, there are some features which affect the performance and the image capture circumstances should be static to ensure objective test results. As an example, the execution time of the auto focus and auto exposure depends on the environment.
SoMA also enables the study of performance trends between test executions. This facility is valuable for early detection of trends in feature development.
Performance testing also enables benchmarking and comparison between different devices. When tests were executed on a sample of commercial devices, significant differences were found. For example, the shutter speed varied from 50 to 225 ms.