Pairwise Test Case Generator

How to use

1. Under “Parameters and values”, write one parameter per line as “Name: value, value, value”. The “Try an example” buttons fill in a website, a mobile app, or a web API
2. If some combinations cannot happen, write them under “Forbidden combinations” (optional)
3. Under “Combinations to cover”, choose pairs (pairwise) or three or more at a time
4. Press “Generate test cases” to see the table of test cases
5. Use “Copy for Excel” to paste into Excel or Google Sheets, or export with “Copy as Markdown” or “Save CSV”

After changing the input, press “Generate test cases” again to rebuild the table. Above the table you see how many test cases there are compared with trying every combination, and how many combinations they cover.

What pairwise testing is

Pairwise (all-pairs) testing keeps the number of test combinations small. Trying every combination multiplies quickly, yet most defects are said to be triggered by a combination of just two conditions. So the table is built so that, for any two parameters you pick, every combination of their values appears in at least one test case.

For example, 4 browsers, 4 operating systems, 2 account types and 3 payment methods make 4 × 4 × 2 × 3 = 96 combinations in all. With pairwise testing, a dozen or so test cases still include every pair.

To also catch defects that need three conditions at once, set “Combinations to cover” to “3 at a time”. There are more test cases, but every combination of values of any three parameters appears. You can choose up to 6 at a time, and choosing the number of parameters gives every combination.

Writing parameters and values

Browser: Chrome, Edge, Firefox, Safari
OS: Windows, macOS, Android, iOS
Account: Member, Guest
Payment: Credit card, PayPal, Bank transfer

• Write one parameter per line. Separate the name from the values with : and the values from each other with ,
• The full-width ：, 、 and ， also work. Spaces around values are removed
• A value may contain : (such as 9:00). Only the first : separates the name
• A parameter can have up to 50 values, and there can be up to 50 parameters. A parameter with a single value is fine
• A name cannot contain =, ≠, , or 、 (or the full-width ＝ and ，), or end with ! (or ！), because a forbidden combination would read them as separators. A value cannot contain |
• Lines starting with # and blank lines are skipped, so you can leave notes
• Values are compared exactly as written. ON and on, or full-width and half-width letters, are different values

This is the same form as the parameters in a model file for PICT, Microsoft's command-line tool, so you can paste the parameter part of a PICT model as is (constraints are written differently).

Writing forbidden combinations

List combinations that cannot really happen, such as “Safari on Windows”, and no test case will include them. Write one per line, joining the conditions that cannot all hold together with ,.

OS = Windows, Browser = Safari
Method = GET | DELETE, Content-Type != None
Browser = Safari, OS != macOS | iOS

• Name = value means that value
• value | value means any of those values
• Name != value (or ≠, or the full-width ！＝) means any other value
• No row meets every condition on a line. The first line above forbids “OS is Windows and Browser is Safari”
• A line with a single condition (Browser = IE) means that value is never used

Pairs that the forbidden combinations make impossible are dropped from what has to be covered and listed under “Combinations that cannot be included”. The third line above means “Safari only runs on macOS or iOS”, so “Safari with Windows” and “Safari with Android” cannot be included.

Forbidden combinations can also rule out pairs you did not write. When B has only the values x and y, with “A = 1, B = y” and “B = x, C = p”, A = 1 leaves only B = x, so A = 1 with C = p cannot be included either. Such pairs are found by checking whether a full test case can still be built.

Example

The “Website” example uses the four parameters above (browser, OS, account, payment) and the forbidden combination “Browser = Safari, OS != macOS | iOS”. Generating pairs gives a table of 15 test cases like this:

No.	Browser	OS	Account	Payment
1	Chrome	Windows	Guest	Bank transfer
2	Chrome	macOS	Member	Credit card
3	Chrome	Android	Guest	PayPal
4	Chrome	iOS	Member	PayPal
5	Edge	Windows	Member	Credit card
(…)
13	Safari	iOS	Guest	Credit card
14	Safari	macOS	Member	PayPal
15	Safari	iOS	Member	Bank transfer

That is 15.6% of the 96 test cases needed to try everything, and it includes all 60 possible pairs of values. Safari appears only with macOS and iOS. PICT also produced 15 test cases for the same model.

Coverage and row order

The “Coverage” column on the right shows what share of the combinations to cover the rows from the first down to that row check. Rows are built so that those near the top include the most new combinations, so if time is short, running the rows from the top checks the most combinations with the fewest test cases.

“Try another arrangement” builds a different table from the same input. The number of test cases may change. The same input and arrangement always give the same table.

Exporting

• Copy for Excel: copies tab-separated text. Paste it into a cell in Excel or Google Sheets to get the table
• Copy as Markdown: a Markdown table for GitHub, Notion and the like. A | in a parameter name and a \ in a name or value are escaped so the table stays intact
• Save CSV: saves test-cases.csv as UTF-8 with a byte order mark, so Excel opens it without garbled characters
• Turn off “Add a No. column to exports” to export without the number column. The on-screen table always shows the numbers

The on-screen table shows the first 1,000 test cases. Copy and save include all of them.

How the table is built, and the limits

Test cases are built one at a time. Each starts from an uncovered combination of the parameter set with the most left, and every other parameter gets the value that completes the most uncovered combinations. Values that would hit a forbidden combination are skipped after checking whether the row could still be finished. This repeats until nothing is left to cover. It is a greedy method in the same spirit as PICT and AETG; several arrangements are tried and the shortest table is shown.

The result is not always the smallest possible. For example, the smallest known table for 13 parameters with 3 values each has 15 test cases, while this tool gives around 19 (PICT gave 20).

When the combinations to cover (for pairs, the value combinations summed over every pair of parameters) exceed 200,000, nothing is generated. Cover fewer at a time or use fewer values.

With many forbidden combinations that interlock, checking whether rows can be completed may take too long, and generation stops. Use fewer forbidden combinations or parameters then. When the forbidden combinations take a lot of checking, fewer arrangements are tried, and the shortest table built so far is shown.

Things to keep in mind

• Pairwise testing finds defects caused by two conditions efficiently. It can miss defects that need three or more conditions together, or a particular order or number of steps. Cover important features three at a time, or add separate tests
• This tool does not pick the values for you (equivalence partitioning or boundary value analysis). Choose representative values yourself, such as “0, 1, 100”
• There is no expected-result column. Add one to the exported table
• Everything you enter is processed only in this browser tab. Nothing is sent or saved, and reloading the page clears it

FAQ

Can I use a PICT model file as is?

You can paste the parameter part as is. PICT constraints (such as IF [OS] = "Windows" THEN [Browser] <> "Safari";), submodels, aliases (|), weights and negative values (~) are not supported. Rewrite constraints as forbidden combinations, such as “OS = Windows, Browser = Safari”.

Can I get fewer test cases?

Pressing “Try another arrangement” a few times sometimes gives a shorter table. Without forbidden combinations, covering pairs always needs at least as many test cases as the product of the value counts of the two parameters with the most values. Two parameters with 10 values each need at least 100 test cases. Use fewer values, or treat the parameters with many values separately.

What are “Combinations that cannot be included”?

Pairs (or larger sets) of values that no test case can include because of the forbidden combinations. They are left out of the coverage figures. If you see one you did not expect, check how the forbidden combinations are written.

How do I make sure a particular combination is included?

This tool cannot place fixed rows first. Add the combinations you must check as extra rows in the exported table.

Does the same input always give the same table?

Yes. When several values score the same, the choice comes from a sequence fixed by the arrangement number, so the same input and arrangement always give the same table. For a different table, press “Try another arrangement”.