Software testing is an investigation of a software product with the intent of finding software bugs and verifying that the program works as expected. This investigation is based on test suites chosen according to certain parameters and specifics of software. In a wider sense, software testing is software quality assurance technique comprising test design, test execution and analysis of test results.
Contemporary software products usually have to be developed in very short terms and at limited project budgets. Programming today is rather craft than art for millions of programmers. Such haste, unfortunately, makes them ignore the necessity of software security assurance, thus exposing end users to undue risk. Software quality control is a considerable aspect of the software development process, for it helps provide security, reliability and usability of the product. There exist multiple approaches to and techniques of software testing nowadays. However, efficient testing of complex software systems is rather a creative process than formal compliance with strict, definite rules.
Unit testing is a software testing level at which the smallest parts of software are tested, for example individual classes or functions. Unit testing is often performed by software developers themselves.
Integration testing is a software testing level at which interfaces between components or subsystems of software are tested.
System testing is a software testing level at which a complete integrated system is tested to evaluate the system’s compliance with its specified requirements. System testing includes alpha testing and beta testing subcategories.
Classification of Testing Types
All the testing types are classified into several categories according to a number of criteria. The following are the most common ones.
By the Object under Test
Functional testing is a software testing type intended to validate that the program complies with functional requirements. Functional testing checks if a software product can correctly solve certain tasks as required by users.
- Wikipedia. Functional testing.
- StackOverflow. Unit tests vs Functional Testing.
- Unit, Integration, and Functional Testing
Performance testing is a software testing type performed to determine how a system performs in terms of responsiveness and stability under a particular workload. Performance tests are executed before and after program optimization in order to find out any changes of performance. If an optimization leads to a performance drop and therefore is unsuccessful, the programmer may refuse using it. In case of a performance gain, its degree is compared to the anticipated one to make sure that the optimization has been done successfully. The purpose of performance testing is to find out any cases of performance increase and decrease in order to avoid faulty modifications.
- Wikipedia. Software performance testing.
- Wikipedia. Load testing.
- StackOverflow. Performance testing utilities for C++.
Load testing is a software testing type performed to determine a system’s behavior under normal, increased, and anticipated peak load conditions. Being performed before the commercial exploitation of the system, load testing helps assure that no sudden performance losses will occur later (in about a year), when the system is loaded with data.
Stress testing is a software testing type that determines robustness of software beyond the limits of normal operation, i.e. under stress conditions like large input data amounts, disk space shortage or execution on a low-performance processor.
Stress testing is also performed on a customized solution and server group to evaluate its ability to serve many users at a time. It is not only the server group which is checked by stress tests but also the influence of software customization on the system performance in general and its fault-tolerance. This testing type requires a network of several computers emulating activity of a group of users.
Stability/endurance/soak testing is a software testing type that determines stability of software during a long testing time under medium (anticipated) workload.
Wikipedia. Software Reliability Testing.
Security testing is a software testing type that evaluates the ability of software security mechanisms to react to system intrusions.
- Wikipedia. Data security.
- Wikipedia. Security testing.
- Bruce Potter, Gary McGraw. Software Security Testing.
Compatibility testing is a software testing type that evaluates the application’s compatibility with a certain computing environment.
- Wikipedia. Compatibility testing.
By the Depth of Knowledge of the System
Black-box testing is a software testing type performed on a software system which can only be accessed through user interfaces or external interfaces which allow another computer or process to connect to the system to run tests on it. This technique is still the most popular in everyday testing practice, but it has a lot of drawbacks. For instance, some errors can only occur in specific conditions and therefore are difficult to find and reproduce.
White-box testing is a software testing type when the tester has a free access to the source code of an application and can write test codes covering the libraries it uses. White-box testing includes methods of code review and software inspection. This methodology allows peeping into the “black box” to focus on the application’s internal implementation which determines its behavior. The main problem with this methodology is that run-time computations are difficult to monitor. White-box tests evaluate the program’s logic, so testing is said to be complete when it tracks all the possible code paths. Even a program of medium complexity may have dozens of thousands of code paths.
By the Product Maturity Phase at Which Testing is Performed
Alpha testing is a software testing type implying simulated or actual operational testing (by developers or end users correspondingly).
Beta testing is a software testing type when limited versions of software are released to a group of people so that further testing can ensure the product has few faults or bugs.
Regression testing is a software testing type performed to ensure that no previously found bugs have come back after any code modifications and bug fixes, as well as to determine the main functionality of the software product. This type of testing is usually performed on every new product version. It is the most important testing phase immediately before the end of the development process, for it is extremely crucial to make sure before the release that not only the main functions work properly but also that no previously detected bugs have been brought into the final version. Being an integral part of functional testing, regression testing guarantees that no modifications brought into the code by bug fixes have caused negative influence on all the other functions of the application.
Smoke testing is a software testing type performed as a set of tests to ensure that the product can be launched successfully. If no errors occur during the launch, the smoke test is said to be passed. If the smoke test is failed, software is sent back to the developer team for revision. The point is that sometimes different components of a program are written by different developer teams and may often appear non-compatible when being assembled into a whole, which makes it pointless to test the product in general.
- Wikipedia. Smoke testing.
- StackOverflow. What is a smoke testing and what will it do for me?
By the Automation Criterion
Manual testing is a software testing type when no software tools are used to run tests and analyze test results. Instead, it is done manually by the tester.
Automated testing is a software testing type when software tools are used to perform tests and analyze test results. Naturally, automated testing is highly useful, as it allows saving time and resources.
It often happens during the development process that new versions with bug fixes are released every day or even several times a day. Smoke tests should be automated first of all because we need to check as soon as possible after every build of a new version that the program can be successfully launched. Automated tests can be completed in just a few seconds, so you can quickly conclude if the build is successful. Being performed by a human tester, the test will take him much more time. Thus, smoke testing automation is an opportunity for the testing department to save incredibly much time.
There exist a number of applications providing test automation features. The most popular are HP LoadRunner, HP QuickTest Professional, HP Quality Center, and TestComplete.
Automation in general does not only helps save development time but also improves security and reliability of software products. Testers will also see significant advantages of this technique: reliability of the testing process and results improves, testing time decreases, and the tester’s work gets less stressful. Automated testing will never replace human of course, but it indeed can simplify the software testing engineer’s work.
Dynamic and Static Code Analysis
As a software product is passing later development phases, the cost of bug elimination may grow exponentially. Static and dynamic analysis tools can help prevent these expenses due to earlier detection of software bugs.
Run-time analysis (dynamic analysis) is a software testing type performed when executing the program. Dynamic analysis reveals defects in the source code as they are occurring. The analysis process can be divided into a number of steps: preparing input data, running a test program launch, gathering the necessary parameters, and analyzing the output data.
Static analysis is a software testing type performed without actual execution of the program. Static analysis enables quick detection of bugs in the source code before it can be executed.
In practice, developers may use both analysis techniques to speed up the development process and testing process, as well as improve the quality of the software product.