Black box testing is investing the system just like any outsider would do, using tools for detecting the attack surfaces and examining the system to check internal information. Without any internal knowledge about the system, testers build an idea about it. Leakage of information is essential to the testers as it helps them in building more idea than they would get otherwise by operating leak-free programs.
In black box testing, the tester examines all the attacked surface, generating a test report for the functionalities which might not be present inside the design. A commonly committed mistake here is not eliminating the debug-only commands before the production. Another is throwing in the last-minute functionalities which aren’t documented properly within the design. Such flaws, being difficult to otherwise notice through white box testing, are brought to notice with the help of black box testing.
Black box testing is generally used at the time when a particular system deliberately uses security by anonymity to help in protecting data, which is usually the matter in the Digital Rights Management (DRM) systems. Software-only DRM isn’t possible for completely securing as the attackers have control over the system in which that DRM software is getting executed. The best that DRM manufacturers can expect is raising the limits for successful attacks so up that a potential attacker would willingly give up. Highly trained reverse-engineering teams usually use black box testing for testing the potency of the used obscurity. This usually needs skills beyond the QA team’s norms and can be very costly. For DRM systems, however, demonstrating the potency of the used obfuscation is very necessary.
In black-box testing, access to the source code is not necessary, although it will help in improving the tests. Black-box testing is built on the responses to a set of inputs fed to the software through selected interfaces, such as-
- User interfaces: GUI, command line;
- Network protocols;
- Data structures such as files;
- System APIs such as system calls and device drivers.
Purposes of Black Box Testing
Some prominent purposes of black box testing are as follows-
- Interoperability testing-Interoperability is basically a subset of conformance testing. Interoperability testing is a practical task in which the final product or its prototype is tried against other industry products.
- Conformance testing-Conformance testing, a category of black-box testing, is a process that aims at validating the conformity of the product against the specifications.
- Performance testing-The third type of testing, performance testing, comes from real-use scenarios of the software. When the software works according to the feature set and with other vendors’ products, testers need to assess if the software is efficient enough in real-life use scenarios. There are different categories of tests for this purpose, including stress testing, performance testing, and load testing.
- Robustness testing-another major black-box testing category is robustness testing or negative testing. From the security perspective, this category is often referred to as the most important category. In negative testing, the software is tortured with semi-valid requests to assess the reliability of the software.
Security Techniques for Black Box Testing
1. Load Testing
The easiest and best-known attack to QA people includes various DoS (Denial of Service) situations. Most of the DoS attacks generally depend on the load. In the case of load testing, the system’s performance concerns are tested using the quick repetition of the test cases and running multiple test cases in parallel.
2. Stress Testing
By preventing access to the needed resources, the stress tests generally alter the operational domain for the SUT. Some instances of the alteration are:
- Speed and size of the available memory
- Speed and size of the available disk
- Total available processors and their processing speed
- Environmental variables
Stress tests are usually executed within a framework of test automation that helps the engineers in running the SUT within a highly controlled environment like a sandbox.
3. Unit Testing
In the case of unit testing, SUT is a module that is used within the real application. The real application logic can be bypassed by implementing parts of the functionality in prototypes when the real implementation is still unavailable or by other replacement implementations when the target is actually a library such as a file parser or a codec.
4. Fault Injection
Traditionally, the term “fault injection” defines a technique of hardware testing wherein artificial defects are entered inside a printed circuit board. The objective here is testing the hardware’s defect tolerance capability or its sensitivity to defects appearing during the manufacturing process or the product lifetime. Fault injections are used for forecasting the hardware’s behavior during the operations. They are even used for guiding the efforts for making that hardware much more powerful against potential flaws.
5. Syntax Testing
Syntax testing is carried out with the objective of verifying if the system executes some input validation over critical interfaces. Several different kinds of anomalies (or errors) can be produced in syntax testing:
- Syntax errors: These errors tend to violate the underlying language’s grammar.
- Delimiter errors: A delimiter marks field separations within a sentence. In case of ASCII-coded languages, these fields are generally letters and characters, and the delimiters are white-space characters (line-feed, tab, space, etc.), other types of delimiter characters (semicolons, commas, etc.), or a combination. The delimiters can be repeated, omitted, multiplied, or substituted by some other typical characters. Paired type delimiters such as braces can be left unbalanced. Unexpected and wrong delimiters are added at the places in which they may not be expected.
- Field-value errors:Field-value errors are illegal fields within a sentence. A field value generally consists of a single range or several dis-articulate ranges of acceptable values.
- Context-dependent errors:Context-dependent errors: context-dependent errors end up violating certain properties within a sentence that cannot be defined by a context-free grammar.
- State dependency error:Every sentence is not acceptable in a software component’s each state. State dependency errors are, for example, correct sentences during incorrect states.
6. Negative Testing
The most common negative testing type is defining the negative test cases like use cases- for example, in case a feature starts implementing a validation functionality, positive tests would include trying a valid username and password. All other things including wrong password, wrong username, are negative testing. Robustness testing is done for trying only the negative tests without caring about any response from SUT.
Robustness testing consists of the following steps, which are very similar to the steps in syntax testing:
- Identify the interface language.
- Define the syntax of the informal language.
- Create a model by augmenting the protocol with semantics such as dynamically changing values.
- Validate that the syntax (the protocol and the resulting model) is complete, consistent and satisfies the intended semantics.
7. Regression Testing
Post-release testing is even called regression testing. This testing requires being very fast and automated. The tests even require being configurable and stable.
Conclusion
This article discusses testing approaches that could be useful to you when integrating security testing into a standard quality assurance process. The aim of black box testing is not to prove that the software is secure but to break software by whatever means available. Follow the above steps for efficient security testing of your products.
Reference source: https://cdn.ttgtmedia.com/searchSoftwareQuality/downloads/Fuzzing_SW_Sec_testing_CH03.pdf