Creating Pairwise Tests using PICT

A primer on using the Pairwise Independent Combinatorial Tool to perform pairwise testing.

This post builds upon the introduction to Pairwise Testing discussed in the preceding entry.

Accelerate Test Suites with Pairwise Tests

Pairwise Testing is a method for efficiently testing combinations of input parameters.

This post outlines how to use the Pairwise Independent Combinatorial Tool to perform pairwise testing on an application with multiple input fields (test vectors) and with a choice of values for each test vector. The test vectors and options are:

• Name: a text field that requires at least three characters
• Email: a text field that requires a valid email address
• Address: a text field that requires at least ten alphanumeric characters
• Payment Method: a drop-down list with the options Credit Card, PayPal, and Cash On Delivery
• Delivery Method: a drop-down list with the options Standard or Express
• Newsletter: a checkbox that indicates whether the customer wants to receive promotional emails

Creating test cases using the traditional approach to cover all possible combinations of these input fields and options would require at least 3 x 3 x 3 x 3 x 2 x 2 = 324 test cases. This would be costly and inefficient to execute.

Creating Pairwise Tests Using PICT

Instead of using the traditional approach, pairwise testing can generate a smaller test suite that covers all possible pairs of test vectors and options.

Pairwise Independent Combinatorial Tool (PICT) is a free command-line tool developed by Microsoft that creates pairwise test suites. PICT takes a model file as input and produces a test suite as output.

The model file for our example application looks as follows. Each line in the model file represents a test vector, followed by a colon and the possible values for that field or option.

Name:3-chars,Invalid,Empty
Email:Valid,Invalid,Empty
Address:10-chars,Invalid,Empty
Payment-Method:Credit-card,PayPal,cash-on-delivery
Delivery-Method:Standard,Express
Newsletter:Yes,No

The model file can be fed to PICT using the following:

pict ./model.txt

The test suite generated by PICT is below, showing a suite size reduction of over 95%!

Name         Email        Address      Payment-Method   Delivery-Method   Newsletter
3-chars      Valid        10-chars     Credit-Card      Standard          Yes
3-chars      Valid        10-chars     PayPal           Express           No
3-chars      Valid        Invalid      Cash-On-Delivery Standard          No
3-chars      Invalid      Empty        Credit-Card      Express           Yes
3-chars      Empty        10-chars     Cash-On-Delivery Express           Yes
Invalid      Valid        Invalid      Credit-Card      Standard          No
Invalid      Valid        Empty        PayPal           Standard          Yes
Invalid      Invalid      10-chars     Cash-On-Delivery Standard          Yes
Invalid      Empty        Invalid      Credit-Card      Express           No
Empty        Valid        Empty        PayPal           Express           No
Empty        Invalid      10-chars     Credit-Card      Standard          No
Empty        Empty        Invalid      PayPal           Standard          No

As can be seen, each test case covers at least one pair of input fields and options not covered by the previous tests.

Considerations

As mentioned in the previous primer, there are factors to be considered when using pairwise tests.

This approach is best used jointly with other testing techniques to increase confidence in the validation quality.