Month / September 2018

by Alex Siminiuc

Use Test Factories for Dynamic Tests

Most of the time, we need normal unit tests for test automation:

@Test
public void testSearch() {
}

A normal unit test is a public method that does not return a value, it does not have parameters and uses the @Test annotation.

The unit test can have a parameter though if it is used with a data provider:

@DataProvider(name = "searchKeywords")
public Object[][] searchKeywords() {
return new Object[][] {
{ "iphone"}, { "ipad"}, { "iwatch"}  };
}

@Test(dataProvider = "searchKeywords")
public void testSearch(String keyword) {
}

What is a dynamic test then?

A dynamic test is a unit test of a test class that has a constructor.

The constructor may have a parameter or more and saves them as class members:

import org.testng.annotations.Test;

public class SampleTest {

private int n;

public SampleTest(int n) {
  this.n = n;
}

@Test
public void testScript() {
  System.out.println("test script: " + n);
}

}

The test class gets an integer parameter in the constructor and saves it in a member of the class.

The testScript() method displays the value of the n class member.

testScript() is a dynamic unit test because it belongs to a test class that uses a constructor with a parameter (or more).

How do you use such a unit test?

A test factory creates objects of the test class.

These objects are then executed in the background:

import org.testng.annotations.Factory;

public class FactoryTest {

  @Factory
  public Object[] createTests() {
    System.out.println("create the tests dynamically!");
    Object[] tests = new Object[10];

    for (int i = 0; i < 10; i++)
      tests[i] = new SampleTest(i);

    return tests;

}

}

The createTests method creates first an array to store of test objects of the SampleTest class.

It then creates multiple SampleTest objects and saves them in the array.

Finally, it returns the array of objects.

This is the result of executing the FactoryTest class:

[RemoteTestNG] detected TestNG version 6.14.2
create the tests dynamically!
test script: 0
test script: 1
test script: 5
test script: 3
test script: 9
test script: 2
test script: 7
test script: 6
test script: 8
test script: 4
PASSED: testScript
PASSED: testScript
PASSED: testScript
PASSED: testScript
PASSED: testScript
PASSED: testScript
PASSED: testScript
PASSED: testScript
PASSED: testScript
PASSED: testScript

===============================================
Default test
Tests run: 10, Failures: 0, Skips: 0
===============================================

We see in these results 10 tests being executed and passing.

The interesting part is that they were not executed in the order they were created.

When would dynamic tests be needed in Selenium test automation?

Let’s assume that we want to automate a test case for the https://www.thebay.com/ site.

If you mouse over the accessories option of the menu, multiple categories are displayed.

bay

We want to verify that when clicking on any of these categories, results are displayed.

We could get started with a test as follows:

import static org.testng.Assert.assertTrue;

import java.util.List;

import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;

public class CategoriesTest {

@DataProvider(name = "accessoriesCategories")
public Object[][] createData1() {
  return new Object[][] {
    { "New Arrivals"},
    { "Best Sellers"},
    { "Online Only"},
    { "Watches"}
  };
}

@Test(dataProvider = "accessoriesCategories")
public void testCategoriesUsingDataProvider(String category) {

  HomePage homePage = new HomePage();
  homePage.open();

  Menu menu = homePage.openAccessoriesMenu();

  ResultsPage resultsPage = menu.selectCategory(category);
  assertTrue(resultsPage.hasResults(), "no results for " + category);

}

}

The categories are defined in a data provider and the test has a parameter for a category.

The test will be executed once for each category.

The test uses 3 classes.

HomePage.java

public class HomePage {

public void open() {
  System.out.println("open home page");
}

public Menu openAccessoriesMenu() {
  System.out.println("open accessories menu");
  return new Menu();
}

}

Menu.java

import java.util.Arrays;
import java.util.List;

public class Menu {

public ResultsPage selectCategory(String category) {
  System.out.println("category selected: " + category);
  return new ResultsPage();
}

public List getCategories() {
  System.out.println("get categories");

  return Arrays.asList( "New Arrivals",
                        "Best Sellers",
                        "Online Only",
                        "Watches");
  }

}

ResultsPage.java

public class ResultsPage {

public boolean hasResults() {
  return true;
}

public HomePage goBack() {
  System.out.println("go back to home page");
  return new HomePage();
}
}

The result of executing the test is below:

[RemoteTestNG] detected TestNG version 6.14.2
open home page
open accessories menu
category selected: New Arrivals
open home page
open accessories menu
category selected: Best Sellers
open home page
open accessories menu
category selected: Online Only
open home page
open accessories menu
category selected: Watches
PASSED: testCategoriesUsingDataProvider("New Arrivals")
PASSED: testCategoriesUsingDataProvider("Best Sellers")
PASSED: testCategoriesUsingDataProvider("Online Only")
PASSED: testCategoriesUsingDataProvider("Watches")

===============================================
Default test
Tests run: 4, Failures: 0, Skips: 0
===============================================

The solution works if the categories for the Accessories option do not change.

But what if they change all the time?

Obviously, we will not be able to use the data provider any longer so the test will change:

@Test
public void testCategoriesWithoutDataProvider() {
  HomePage homePage = new HomePage();
  homePage.open();

  Menu menu = homePage.openAccessoriesMenu();

  List < String > categories = menu.getCategories();

  for (String category: categories) {
    ResultsPage resultsPage = menu.selectCategory(category);
    assertTrue(resultsPage.hasResults(), "no results for " + category);

    homePage = resultsPage.goBack();
    menu = homePage.openAccessoriesMenu();
  }

}

The test is more complicated than before.

It gets all categories of the menu and then iterates through them. For each category, the test selects the category, verifies that there are results and then goes back to HomePage.

The result of executing the test is below:

[RemoteTestNG] detected TestNG version 6.14.2
open home page
open accessories menu
get categories
category selected: New Arrivals
go back to home page
open accessories menu
category selected: Best Sellers
go back to home page
open accessories menu
category selected: Online Only
go back to home page
open accessories menu
category selected: Watches
go back to home page
open accessories menu
PASSED: testCategoriesWithoutDataProvider

===============================================
Default test
Tests run: 1, Failures: 0, Skips: 0
===============================================

The test does pretty much the same thing as before.

But instead of having one test for each category, we have 1 single test for all categories.

If the test fails, it will not be clear which category has a problem.

So this is what we want to fix.

We will simplify the test class so it does only the selection of the category:

import static org.testng.Assert.assertTrue;

import java.util.List;

import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;

public class SelectCategory {

private String category;

public SelectCategory(String category) {
this.category = category;
}

@Test
public void selectCategory() {
  HomePage homePage = new HomePage();
  homePage.open();

  Menu menu = homePage.openAccessoriesMenu();

  ResultsPage resultsPage = menu.selectCategory(category);
  assertTrue(resultsPage.hasResults(), "no results for " + category);

}
}

And we will use a test factory class that gets all categories and creates multiple objects for the SelectCategory class:

import java.util.List;

import org.testng.annotations.Factory;

public class SelectCategoryFactory {

@Factory
public Object[] testCategories() {
  System.out.println("start test factory");

  HomePage homePage = new HomePage();
  homePage.open();

  Menu menu = homePage.openAccessoriesMenu();

  List < String > categories = menu.getCategories();

  closeBrowser();

  Object[] tests = new Object[categories.size()];

  for (int i = 0; i < categories.size(); i++)
    tests[i] = new SelectCategory(categories.get(i));

  System.out.println("end of test factory");

  return tests;
}

private void closeBrowser() {
  System.out.println("close site");
}

}

The result of executing the test factory shows again 4 tests passing:

[RemoteTestNG] detected TestNG version 6.14.2
start test factory
open home page
open accessories menu
get categories
close site
end of test factory
open home page
open accessories menu
category selected: Watches
open home page
open accessories menu
category selected: Best Sellers
open home page
open accessories menu
category selected: New Arrivals
open home page
open accessories menu
category selected: Online Only
PASSED: selectCategory
PASSED: selectCategory
PASSED: selectCategory
PASSED: selectCategory

===============================================
Default test
Tests run: 4, Failures: 0, Skips: 0
===============================================

The test factory allows the initial test to be changed so that it works for dynamic categories. It also allows executing a test for each category.

by Alex Siminiuc

How good is your Selenium Java code?

code-review

 

You probably know who Martin Fowler is.

He coined the term Page Object Model sometime in 2013:
https://martinfowler.com/bliki/PageObject.html

Yes, the page object model that you use in your Selenium test automation projects.

His contribution to software development goes way beyond the page object model since he wrote many useful books on topics such as code refactoring and design patterns.

He also said that

Any fool can write code that a computer can understand.
Good programmers write code that humans can understand.

This is what I want to focus on today, on the code written by good programmers.

Good programmers write code that other programmers can understand but also can change and maintain.

This is what we should aim at in our Selenium projects.

Our code should also be easy to understand, change and maintain.

 

How do you know if your code does all these?

What are the rules that your code should follow so that it is good?

Let’s see.

 

Write Clean Code

The code should be clean.

This means a lot of things.

1. The code should use clear names

Variables should have names that explain their purpose.
Methods should have names that either explain what the method does or the result returned by the method.

2. Use problem domain names

If you do automation for an e-commerce site, make sure that you have classes for concepts such as

  • product
  • user
  • basket
  • cart
  • order
  • invoice

3. Classes should be small

A class should contain an average of less than 30 methods.

4. Methods should be small

Methods should not have more than an average of 30 code lines.

5. Do one Thing

This applies to both classes and methods.
If a method does more than one thing, consider splitting it in 2.
If a class has more than one responsibility, consider breaking it in more classes.

6. Don’t Repeat Yourself 

Any duplication, inside a page object class, a page method, a test class or a test method should be avoided.

7. Explain yourself in code

Write code that is self-explanatory, that is so easy to understand so no comments are needed.

8. Make sure the code formatting is applied

Code formatted correctly is easier to read by other developers.

9. Use Exceptions rather than Return codes

If a method cannot fulfill its purpose, instead of returning an obscure error code,
throw an exception since the code is in an abnormal state.

10. Don’t return Null

There are many ways of avoiding returning null.
You can use Optional introduced in Java 8.
Or you can return an empty list.

Write Secure Code

1. Make class final if not being used for inheritance

Making the class final ensures that it is not extended.

All page classes and page element classes should be final.

2. Avoid duplication of code

3. Limit the accessibility of packages,classes, interfaces, methods, and fields

Parent classes (base page class, base test class) should be abstract.

All methods of a parent class should be declared as protected since they should only be used in the child classes.

Only a small number of methods of any class should be public.

If a class should be used only by other classes in the same package, use the default access modifier.

4. Validate inputs (for valid data, size, range, boundary conditions, etc)

Any public method should have its parameters checked for validity.

5. Avoid excessive logs

If you are logging tracing information for all page classes and their methods, when you run the whole suite of tests, you will get excessive logs that are very difficult to read.

This becomes even worse if the automated tests are run in parallel on different virtual machines.

6. Release resources (Streams, Connections, etc) in all cases

Consider using try/catch with releasing resources.

7. Purge sensitive information from exceptions (exposing file path, internals of the system, configuration)

8. Do not log highly sensitive information

9. Make public static fields final (to avoid caller changing the value)

10. Avoid exposing constructors of sensitive classes

Use a factory method instead of the constructor to create objects.

General Rules

1. Use checked exceptions for recoverable conditions and runtime exceptions for programming errors

2. Favor the use of standard exceptions

3. Don’t ignore exceptions

4. Check parameters for validity

5. Return empty arrays or collections, not nulls

6. Minimize the accessibility of classes and members

7. In public classes, use accessor methods, not public fields

All fields of a class should be private so that objects cannot be changed (stay imutable).
Set methods are bad for the same reason since they make an object mutable.

Get methods should be used to expose fields outside of the class.

8. Minimize the scope of local variables

Declare the variable just before you need it and not at the beginning of the method.

9. Refer to objects by their interfaces

Also called program to an interface.

10. Adhere to generally accepted naming conventions

11. Use enums instead of int constants

SortOrder.PriceAscending is always more clear than 1.

12. Beware the performance of string concatenation

Use StringBuilder class instead.

13. Avoid creating unnecessary objects

 

This is obviously an incomplete list.

But it should get you started on focusing more on the quality of your code.

Because, good programmers write good code and you want to be a good programmer.

Don’t you?