Common Automation Mistakes

Watch out for these common errors when writing test code:

  • Hard-coded paths Tests often need external files during test execution. The quickest and
    simplest method to point the test to a network share or other location is to embed the path in the
    source file. Unfortunately, paths can change and servers can be reconfigured or retired. It is a
    much better practice to store information about support files in the TCM or automation
  • Complexity The goal for test code must be to write the
    simplest code possible to test the feature sufficiently.
  • Difficult debugging When a failure occurs, debugging should be a quick and painless
    procedure—not a multihour time investment for the tester. Insufficient logging is a key
    contributor to making debugging difficult. When a test fails, it is a good practice to log why the
    test failed. “Streaming test failed: buffer size expected 2048, actual size 1024” is a much better
    result than “Streaming test failed: bad buffer size” or simply “Streaming test failed.” With good
    logging information, failures can be reported and fixed without ever needing to touch a
  • False positives A tester investigates a failure and discovers that the product code is fine, but a
    bug in her test caused the test to report a failure result. The opposite of this, a false negative, is
    much worse—a test incorrectly reports a passing result. When analyzing test results, testers
    examine failures, not passing tests. Unless a test with a false negative repeats in another test or
    is caught by an internal user during normal usage, the consequences of false negatives are bugs
    in the hands of the consumer.