Model-driven Test Case Generation

An approach, where the test cases are generated in parts or completely from a model is called Model-driven Testing. In general, a distinction is made between

• System specifications, which model the functional behavior of the SUT, i.e. the system under test and
• Usage models that model the usage behavior of the future users of the system.

Test cases, which are generated from a system specification are often used in the so-called component or unit test. Usage models are mostly applied for the generation of test cases for the system or acceptance test.

The widespread V-Model clarifies this relation by distinguishing between the development and the test of a system:

Statistical Test Case Generation

With the help of statistical usage models, so-called Markov chains, individual test cases or complete test suites are automatically derived by simple push button operations. Markov chains are graphical models and represent

• usage states for modelling the user behavior during the interaction with the system, as well as
• state transitions to specify the reaction of the system on a user's interaction.

For example, the input of a search item in the search field of an Internet browser is an interaction of the user with the browser and the output list of web pages that are fulfilling the search criterion represents the expected reaction of the system. Both usage states <input of a search item> and <output of a list with web pages> as well as the state transition between these usage states that is triggered by the interacting user after pressing the <Enter> key are described in the Markov chain.

Usually there are many alternatives to reach possible successor states from a given usage state. For example, instead of pressing the <Enter> key in our example a different action may be performed, such as switching to another window by using the mouse. This will take the user to another usage state.

The probability that a particular action can be executed is called transition probability between the usage states. By adjusting the probability values it is very easy to specify varying usage behavior for different user classes. In this way, the test engineer can automatically create distinct test cases for different system users. Test cases are derived by traversing the Markov chain between to specific start and end states by considering the probabilities of the selected usage profile.

Test-driven Agile Simulation (TAS)

In the TAS approach test cases that are otherwise used in the latter system test can be used for an early validation step of the involved system and usagage models. This is done by simulating a given system specification and running test cases at the model level. The method also implies the automatic generation of executable simulations and of test cases from corresponding models, which can be executed in our VeriTAS simulation framework.

Research Issues

• tool development for sceanario-based statistical testing based on UML, UTP and TTCN-3
• tool development for test-driven agile simulation techniques based on the VeriTAS simulation framework
• development of hybrid simulation models for the prospective health technology assessment (ProHTA)

• Anatoli Djanatliev
• Winfried Dulz
• Vitali Schneider

• A. Djanatliev, W. Dulz, R. German, and V. Schneider. VeriTAS - A Versatile Modeling Environment for Test-driven Agile Simulation. Proceedings of the 2011 Winter Simulation Conference, 2011.
• W. Dulz. A Comfortable TestPlayer for Analyzing Statistical Usage Testing Strategies. Proceedings of the 6th ICSE/ACM Workshop on Automation of Software Test (AST '11), 2011.
• I. Dietrich, F. Dressler, W. Dulz, and R. German. Validating UML Simulation Models with Model-Level Unit Tests. Proceedings of the 3rd International ICST Conference on Simulation Tools and Techniques (SIMUTools '10), 2010.
• W. Dulz, R. German, Stefan Holpp and Helmut Götz. Calculating the usage probabilities of statistical usage models by constraints optimization. Proceedings of the 5th ICSE/ACM Workshop on Automation of Software Test (AST '10), 2010.
• W. Dulz, S. Holpp and R. German. A Polyhedron Approach to Calculate Probability Distributions for Markov Chain Usage Models. Electr. Notes Theor. Comput. Sci. 264(3), 2010.
• W. Dulz. On-the-fly Interpretation of Test Cases in an Automatically Generated TTCN-3 Test Suite. Evaluation of Novel Approaches to Software Engineering, in Communications in Computer and Information Science, Volume 69, I, 17-30, 2010
• S. Siegl, W. Dulz, R. German, G. Kiffe. Model-Driven Testing based on Markov Chain Usage Models in the Automotive Domain. Proceedings of the 12th European Workshop on Dependable Computing (EWDC 2009). Toulouse, France, 14-15 May 2009.
• F. Prester and W. Dulz. markov@.mzT - .model centric Testing (.mzT) using markov chain theory. Proceedings of the embeddedWorld 2009 (eW2009), 2009.
• M. Beyer, W. Dulz and K.-S. Hielscher. Performance Issues in Statistical Testing. Proceedings of 13th GI/ITG Conference Measuring, Modelling and Evaluation of Computer and Communication Systems (MMB '06) , 2006.
• M. Beyer and W. Dulz. Scenario-Based Statistical Testing of Quality of Service Requirements. International Workshop "Scenarios: Models, Transformations and Tools", Revised Selected Papers, Springer LNCS 3466, 2005.
• W. Dulz and F.-H. Zhen. MaTeLo - Statistical Usage Testing by Annotated Sequence Diagrams, Markov Chains and TTCN-3. Proceedings of the Third International Conference on Quality Software (QSIC '03) , 2003.
• M. Beyer, W. Dulz and F.-H. Zhen. Automated TTCN-3 test case generation by means of UML sequence diagrams and Markov chains. Proceedings of the 12th Asian Test Symposium (ATS '03), 2003.