Automating Software Testing — When to Create Automated Test Cases

It's time to do automated testing — an important type of software testing. But how do you decide which test cases to automate?

Automating all of your tests may sound appealing at first, but that's both difficult and impractical. Some tests are better suited for manual testing while some make excellent candidates for automated testing.

So, you might be wondering...

What Test Cases Can Be Automated? 

Here are seven tips for deciding what test cases can be automated. 

1. How often is the test run? Would you benefit from having it run more often?

A common test, like making sure you can login, is simple and often easy to automate. It also would be beneficial to find out right away if you cannot login, and could be run after every single build (as in a smoke test). A test that only needs to be run once before release isn't likely to be worth automating.

2. How much data needs to be entered for the test? (Or, how tedious is the test)?

Adding one thousand entries to your database is a pain to do manually. But it's no sweat for an automated test. In general, the more data that's entered, the better the test automation candidate.

3. Is the output of the test easily measured?

If you can easily tell if a test succeeded or failed, then it's a good test to automate. If you need to cross-reference several items to make the call, it will be more difficult to automate the test. 

4. Is the output of the test purely objective?

If the tester is required to make sure a result is exact, an automated test can usually do that. If, for example, the tester is required to make sure an image looks "good" or even "readable," an automated test will have a very difficult time verifying the image.

5. Does the UI around the area being tested change a lot from version to version?

An automated testing tool can handle some changes to controls. But it likely could not handle the moving of a field to a completely different dialog. You'd have to modify the automated script to compensate for it.

6. Does the test utilize any customized controls?

If a test uses ordinary buttons, edit boxes, combo boxes, or grids, it can be an automated test. If you have custom controls, it will be difficult. (Not that custom controls cannot be automated. But you may get better results automating with the more standard parts first.)

7. Does the test invite exploration of corner cases or improvisation?

An automated test can only do what it is told. It is not creative and won't explore corner cases.

Learn more about different types of software testing >>

What Should Be Automated in Software Testing? 

So, test cases can (and should be automated) if:

• Tests are used repeatedly.
• Tests involve a lot of data entry.
• Tests clearly pass or fail. 
• Tests deliver an exact result.
• Tests use consistent UI and regular controls.
• Tests are only to do what they're told — not check anything else. 

And What Test Cases Cannot Be Automated?

Only certain types of test cases can be automated. Subjective test cases — test cases that are not testing a clear function — will still need to be done manually.

Here are some examples of test cases that cannot be automated:

• Exploratory tests
• UX tests
• UI tests
• API tests

And that's okay. The point of creating automated test cases is to expand test coverage and help your testers test more in less time.

How to Manage Automated Test Cases

The above are just guidelines. Even if your test meets all the guidelines for a good candidate for automation, it may still be difficult to automate the particular test. 

Automated testing is just one piece of the puzzle.

Automating tests will save you time. But there will always be the need to run some tests manually. And successfully managing both forms of testing is critical to your product's success.

Learn more best practices to help you consolidate software testing without compromising delivery.

Automated testing best practices

Note: This post was originally published on July 21, 2009 and has been updated for accuracy and comprehensiveness. 

Related Resources

• API Performance Testing Tools: JMeter, Taurus, and BlazeMeter