Testing your application

Recently I finished a research about how to test an application. This was needed to have clear guidelines on how you should test your application and what kind of tools could be used for accomplishing that. The result of this research is spread out into the following paragraphs.

What is testing?

The definition of testing

To keep it simple testing is making sure your application does what you expect it should do. This can be determined in different ways:
- Unit isolation testing
- Unit integration testing
- Integration testing
- Acceptance / functional testing

Unit isolation testing
Particularly useful when testing the logic of your application, like calculations etc. There shouldn’t be a lot of these kinds of tests in your code base. In other words, Mock tests.

Goal
The goal of this kind of test should be making sure the logic you have written doesn’t break and produces the right outcome.

Unit integration testing
Used for testing your logic in its context. So only use mocks if it’s really necessary (i.e. for a web service, network resource etc.)

Goal
The goal of this kind of test should be making sure the logic you have written in combination with the context it is living in is producing the right out come and doesn’t brake.

Integration testing
Useful for testing your whole application through all layers so you actually know that your application layers operate properly when joined together. The use of mocks should be prevented here.

Goal
The goal of this kind of test should be making sure the application as is working correctly when all layers are put together.

Acceptance / functional testing
These kinds of tests are almost the same as an integration test but more focused on the end user driving it.

Goal
The goal of this kind of test should be making sure all functional requirements are met.
All these kind of tests can be done in an automated way. Of course there should be some sanity checks done during the Acceptance / functional testing. But most scenarios can be automated.

How should I test?

When to use ‘Unit isolation testing’
Using Unit isolation testing is useful to find out quickly where exactly the problem occurred when your tests fails. Generally these kinds of tests apply only on the service layer of your application where the logic of your application resides. Writing a mocked DAO isolation test doesn’t really make sense because writing a test for a DAO the goal is to know if the data can be retrieved from the data store as you expect it would be retrieved. Or knowing that the data put in is actually stored.

So in general use this one only if you have a class that has some logic that is worthy to test.

When to use ‘Unit integration testing’
These kinds of tests are very important because they actually test your code in the context it’s going to be used. It will test your logic with some real data preferably. Less mocks the better making it a better test because it’s actually testing your real code. Actually these kinds of tests kind off overrule the Unit isolation testing because it’s doing almost the same but closer to reality. These tests in general have more value than an isolated test because they are testing the real thing. It also lets you know when you break something in the code base. The Unit isolation testing is actually the start of these kinds of tests. I would recommend extending your isolation test into a unit integration test instead of writing a new one. But be careful with this because that doesn’t apply to all of the Unit isolation tests. It’s case specific. There is no general rule for doing this. It’s more a decision you should take when you are about to make such a test. Think about the following things:
- How much effort does it cost me?
- Does it give me more assurance?
- What are the risks?
- Do I expose too much?

Answering all these questions will give you a rough answer so you can decide to create a separate test or an extended one.

So in general this could be used when i.e. implementations of interfaces you use are available and you want to test the real thing and making sure your code base is stable.

When to use ‘Integration testing’
These kinds of tests should actually only be run on some continuous build server and not on your local machine to determine if you made the code base unstable. I would say that these kinds of tests are run when the application is actually running so you know that your code actually works when deployed. This is more like a live test not only testing your code but also the configuration of the application. A good example of an integration test would be testing your web services in an automated way, doing the actual web service calls exactly the way they are going to be made when your application is used in production.

So in general always do this to make sure your application actually still runs when you deploy it.

When to use ‘Acceptance / functional testing’
Actually most of these kinds of tests should be done by a separate test team. They have a more objective look on the application than the developers that worked on it. You could write some automated tests that use i.e. the GUI and write some common use cases for your application. This will cause the test team to focus on only specific things that can go wrong. It also provides you with some valuable information that use cases where you wrote tests for, actually still work and have the desired outcome.

So in general I see this more as an extra thing to make it all much nicer and of course you will be able to notice problem in much earlier stage. This does save time eventually and is recommended.

Which tools to use for testing

For writing tests in general people use JUnit. When using the Springframework note that this framework also provides some handy classes to use as your base class for testing. And of course the famous EasyMock is used a lot for creating mock objects in an easy way. Actually there isn’t much to tell about the ‘normal’ testing of your code if it comes to tools.

When it comes to integration testing there are some tools you could use:
- Selenium
- Cargo
- JMeter
- MultiThreadedTC

Selenium
This tool is for automated GUI tests. You can record your actions in your browser and create a JUnit test from it and alter it accordingly.

Website:
http://selenium.openqa.org/

Cargo
To be able to start-up an application server in your tests you can use this tool for the job. You can start-up your application server and deploy your application and run test on it. This can be included in your automated build environment so you know your application actually still works correctly when you plan to deploy it.

Website:
http://cargo.codehaus.org/

JMeter
To load test your application you can use JMeter from Apache. This is something that should be run for every release actually to see if your application still performs the same.

Website:
http://jakarta.apache.org/jmeter/index.html

MultiThreadedTC
MultithreadedTC is a framework for testing concurrent applications. It features a metronome that is used to provide fine control over the sequence of activities in multiple threads.

Website:
http://www.cs.umd.edu/projects/PL/multithreadedtc/index.html