Creating network errors for testing a distributed system

Question

I am developing a Java library for communication over HTTP, and I want to check its reliability and performance in case of network problems such as packet loss, high latency, low bandwidth and congestion. I use Apache httpclient library to make connections from the client side and Java own com.sun.net.httpserver.HttpServer to start the HTTP server.

Are there libraries that do such things, or should I use them myself? I guess I could try connecting my own org.apache.http.conn.scheme.SchemeSocketFactory on the client side and simulate some of the problems mentioned above, but I would rather use something that already works :-)

This is similar to the question of creating TCP network errors for unit testing , but I am looking for Java-related solutions on Linux. I looked at the click that was suggested for this question, but I'm not sure if it can provide what I'm looking for.

Answer 1

Since the Java API does not have access to the low-level network API, emulating it will be difficult.

Alternatively, the Apache HTTP Core Kernel Component Library may have something to help you simulate latency and data loss, and possibly bandwidth problems. The library has a function for entering your own input and output buffers. Documentation can be found in Advanced HTTP Kernel Features

Enter your own buffer

• Delay may be a slight delay introduced in reading and writing
• Data loss is discarded bytes. This is not packet loss in the true sense, since duplicate resubmission is not performed.
• Bandwidth can be limited by setting read / write and hide operations.
• Traffic is difficult to model, except for all the slowdown in reading and writing to the buffer.

You can also see TCPProxy available in the Grinder project. I did not look at it in detail. The documentation shows EchoFilter , but I think it should be possible to expand the filters to delay sending and receiving bytes in the stream, but it makes sense to use a proxy server to simulate network problems. Thus, your client and server remain blissfully unaware of testing.

In both cases, modeling network problems seems to be the best possible for Java. Or you can configure a local firewall and configure it to intercept and play the packets it receives.

Hope this was helpful.

Answer 2

If you are not testing system integration, IE launches a full stack with external servers, etc., then using bullying tools is what you are looking for. They allow you to record the behavior of the library you are working with. This will allow you to run library code that does not require verification.

Using something like mockito, or if necessary PowerMock , you can tell the library to throw exceptions when the method you call is called.

Example:

import static org.mockito.Mockito.mock import static org.mockito.Mockito.when ... @Test(expected=HttpException.class) public void invockationThrowsHttpException() { ... HttpClient httpClient = mock(HttpClient.class) when(httpClient).executeMethod(args...).thenThrow(HttpException...) underTest.invokeCodeUnderTest(args...) ... } 

The above example assumes JUnit4. The Mockito website has a pretty good tutorial. PowerMock is used if you need to scoff at the fact that mockito is not mocking (for example, static or final methods).

As an aside, I noticed that the Apache HTTP Client is at the end of life and that they recommend switching to the Apache HTTP components instead. If your project is still in its early stages, you can now switch it.

Answer 3

Instead of trying to implement network errors (in your case, this could mean changing the httpclient library or simulating errors and delays on the server), you should consider using a WAN emulator, such as WANEM . In my opinion, this would be a simpler and more acceptable solution.