Best Practice Rules for Test Automation

Best practice rules for JUnit 4

MUST READ THE SECTION BEFORE COMMITTING AUTOMATION CODE

Rule. Recommend to follow Java code conventions
Rule. Don't miss assertions. One test method should include at least one assertion, otherwise the test will always pass.
Rule. Don't add assertions in @Before/@BeforeClass/@After/@AfterClass methods.
Rule. Use assertions with an informative message.
Bad

assertEquals(expected , actual);
assertNull(actual);
assertNotNull(actual);
assertTrue(actual);

.

Good

assertEquals("actual should equals expected", expected , actual);
assertNull("actual should be null", actual);
assertNotNull("actual should not be null", actual);
assertTrue("actual should be true", actual);

Rule. Don't use assertTrue wrongly.
Bad

assertTrue("actual should be same with expected", expected == actual);
assertTrue("actual should equals expected", expected.equals(actual));
assertTrue("actual should be null", actual == null);
assertTrue("actual should not be null", actual != null);

Good

assertSame("actual should be same with expected", expected, actual);
assertEquals("actual should equals expected", expected, actual);
assertNull("actual should be null", actual);
assertNotNull("actual should not be null", actual);

Rule. Use assertArrayEquals instead of assertEquals for array data.
Bad

assertEquals("actual[0] should equals expected0", expected0, actual[0]);
assertEquals("actual[1] should equals expected1", expected1, actual[1]);
assertEquals("actual[2] should equals expected2", expected2, actual[2]);

Good

String[] expected = {"0", "1", "2"};
String[] actual = someMethod();
assertArrayEquals("actual should equals expected", expected, actual);

Rule. Avoid negation in an assertTrue or assertFalse test.
Bad

assertTrue("not empty", !r.isEmpty());

Good

assertFalse("not empty", r.isEmpty());

Rule. Don't catch unexpected exceptions.
Bad

public void testCalculation() {
    try {
        deepThought.calculate();
        assertEquals("Calculation wrong", 42, deepThought.getResult());
    }
    catch(CalculationException ex) {
        Log.error("Calculation caused exception", ex);
    }
}

Good

public void testCalculation() throw CalculationException {
    deepThought.calculate();
    assertEquals("Calculation wrong", 42, deepThought.getResult());
}

Rule. Avoid too complex test. Simple test is easy to understand and maintain. One test should not contain too many asserts. Consider breaking the test scenario into multiple, shorter test scenarios.
Rule. Avoid external dependencies. Others can run test by a single command or a one button click without extra configurations. e.g. Data file should be placed in the svn with test code, so that newcomers don't need to manually prepare it.
Bad

public void testFile() {
    String testFile = "C:\\test\\test.odt";
    //Do test against on testFile
}

Good

public void testFile() {
    // test.odt is placed with test code together. Given a relative path, prepareData will return the file's real absolute path.
    String testFile = prepareData("test.odt");
}

Rule. Make test method indepdent, not affect or be affected by others. Test methods can be executed in any particular order. So do necessary setup/teardown in @Before/@BeforeClass/@After/@AfterClass method to avoid side effects.
Rule. Log the generated data when you use random.
Rule. Name tests properly. "testLoadFileFromServer" is better than a simple word "test".
Rule. Consider parameterized test if you want run the same steps against a bunch of different test data.Use @RunWith(value = Parameterized.class) on class to enable parameterized test. Use @Parameters annotation on one static method to generate a collection of parameters. Each parameter will be passed into class constructor during test.
Example

 
@RunWith(value = Parameterized.class)
public class ParameterizedTest {
 
    private int number;
    public ParameterizedTest(int number) {
        this.number = number;
    }
 
    @Parameters
    public static Collection<Object[]> data() {
        Object[][] data = new Object[][] { { 1 }, { 2 }, { 3 }, { 4 } };
        return Arrays.asList(data);
    }
 
    @Test
    public void testSteps() {
        // use number to do test
    }
}

Best practice rules for GUI testing

Rule. Don't directly use shortcut keys. Make them as an customized shortcut. Different platforms have different shortcuts for the same function.
Rule. Don't add unnecessary sleep.
Bad

 
   //... some steps
   control.waitForExistence(5, 1); // The line is not necessary. 
   //sleep(5); // The line is not necessary. 
   control.click(); // The method impliedly does several times of sleep to wait the control to exist.

Good

 
   //... some steps
   control.click();

Rule. Test testing codes on all platforms before committing. OpenOffice may have different behaviors on different platforms.
Rule. Don't assert if a control exists, if the next operation imply the control must exist.
Bad

 
   //... some steps
   assertTrue("Control exists", control.exists(5, 1));  // The line is not necessary. 
   control.click(); // click method imply the control must exist, so you have add the verification point impliedly.

Good

 
   //... some steps
   control.click();