ISTOCK
Researchers from HSE University and the Russian Technological University (RTU MIREA) have developed an intelligent system to automate software testing on a variety of platforms. Its computer vision feature is capable of recognising elements in a graphical user interface even after a redesign. The details are published in the Journal of the Siberian Federal University.
Testing is a key stage in software development; no software product is complete without it. The role of software testers is to make sure that a new software product runs properly and detects any bugs or defects. To do this, testers log into a website or application and check its functions following a certain sequence of steps. In industrial automated testing, testers often write a test script and then need to update and relaunch it manually every time changes are made to the original software product.
A variety of tools are used for testing websites and mobile applications, and there are also tools specifically designed for testing iOS and Android OS. The need to use different programming languages and duplicate the same script for different platforms can double testers' workload.
Researchers from RTU MIREA and the HSE Faculty of Computer Science have developed an algorithm that makes it possible to use a single testing tool on all systems, whether smartphones or websites.
The tester loads a test script that executes a testing cycle. The algorithm scans the software screen, recognises graphical interface elements, and simulates interaction with them. All actions are performed automatically to mimic a human tester. Once the end of the script is reached, the intelligent system generates a report and ends the process.
We propose using AI such as neural networks and robots for software testing to reduce the need for manual intervention, speed up and simplify the process, and thus ease the tester's workload
Applications often get redesigned, eg by changing the colour of buttons, the radius of curves, or the spaces between different elements. Each update normally requires a new test script to be developed. But our intelligent system can adapt to such changes. The neural network's machine learning and computer vision make it possible to recognise user interface elements regardless of their graphic design. Then there is no need for human testers to be involved.
Vladimir Boyko
Research author, doctoral student at RTU MIREA
The proposed testing tool can be used with any software product that has a graphical interface to automate routine and repetitive tasks in programming.
IQ
Sergey Zykov
Professor, HSE Faculty of Computer Science