As we saw earlier, syntax testing is a special data-driven technique, which was developed as a tool for testing the input data to language processors such as compilers or interpreters. It is applicable to any situation where the data or input has many acceptable forms and one wishes to test system that only the ‘proper’ forms are accepted and all improper forms are rejected.
Applications of Syntax Testing
Traditionally, the following are the applications of syntax testing:
- Command-Driven Software: This is the most obvious application and probably the most common. If a system is mostly command driven, then much of its testing can be organized under syntax testing.
- Menu-Driven Software: The popular alternative to command-driven software is menu-driven software, in which actions are initiated by selecting from choices presented in a menu. However, menu-driven or not, there are still data fields to enter and those fields have a syntax against which syntax testing is effective. It’s usually very easy, though, because the syntax tree tends to be shallow
- Macro Languages: Many commercial software packages for PCs have a macro language, also called a scripting language. This is a programming language that can be used to automate repetitive operations. These languages are often implemented in commercial packages not just for the users’ convenience but because they are a neat way to implement a lot of complicated features, such as mail-merge in a word processor. In their best form, these are full-blown, albeit specialized, programming languages with formal specifications (often in BNF or the equivalent). Non-commercial software or software that serves a narrow segment of an industry*also may have a macro language, but usually, it’s not as clearly defined or implemented as the commercial (horizontal) packages. Such languages, if they are part of an application, warrant syntax testing.
- Communications: All communications systems have an embedded language. It is the language of the format of messages. Even telephone exchanges use language to communicate with each other. But proper telephone numbers, local, long distance and international, have a very formal syntax called a number plan. Every message used to communicate must have a format (i.e., syntax) that must be parsed.
- Database Query Languages: Any database system has a command language used to specify what is to be searched and what is to be retrieved. The simplest ones allow only one key. The more mature systems allow boolean searches based on user-supplied parameters, which we recognize as being predicates. Such predicates obviously have a formal syntax and semantics, and should, therefore, be treated to syntax testing.
- Compilers and Generated Parsers: The one place syntax testing should not be used, especially dirty syntax testing, is to test a modern compiler. This might seem perverse and strange, but a tester should never repeat the tests previously done by another. Modern compilers have a parser, of course. But that parser is generated completely automatically by use of a parser generator given a formal definition in something like BNF. Lexical analyzers are also generated automatically from formal specifications. From a testing viewpoint, there’s not much that syntax testing, even when fully automated, can do to break such lexers and parsers. Before you spend a lot of effort on syntax testing (even if automated), look at the application and how it has been implemented. If a generated lexer and parser are used or planned, you’re unlikely to have great success in using syntax testing; those kinds of bugs have been totally prevented. The only thing left to test is semantics, which is often done by another technique such as domain testing.
Limitations
The biggest potential problem with syntax testing is psychological and mythological in nature. Because design automation is easy, once the syntax has been expressed in BNF, the number of automatically generated test cases measures in the hundreds of thousands. Yet, as in the case of generated parsers, such tests may be no more cost-effective than trying every possible iteration value for a loop. The mythological aspect is that there is great (undeserved) faith in the effectiveness of keyboard-scrabbling or monkey testing. Monkey Testing is just pounding away at the keyboard with presumably random input strings and checking the behaviour. Though amateurish software can still be broken by this kind of testing, it’s rare for professionally created software today. However, the myth of the effectiveness of the wily hacker doing dirty things at the keyboard persists in the public’s mind and in the minds of many who are uneducated in testing technology. Another caveat is that syntax testing may lead to false confidence, much akin to the way monkey testing does.
Conclusion
One major benefit of syntax testing comes from the assurance that there are no misunderstandings about what are legal data and what is not. When a formal syntax description is written out, such problems will surface even before the testing begins. This is another example in which the process of designing and creating test cases helps to prevent errors. Ideally, the formal syntax should be used to specify the system in the first place. The applications and limitations specified above may prove beneficial to adopt syntax testing.
Give us 30 minutes and we will show you how many millions you can save by outsourcing software testing. Make Your product quality top notch. Talk to us to see how