Getting consistent logs while running tests

Question

We used testng with java to perform integration tests for our code. We executed a listener to run the test as follows: -

 public class TestExecutionListener implements IInvokedMethodListener { @Override public void beforeInvocation(IInvokedMethod iInvokedMethod, ITestResult iTestResult) { System.out.println("Testing : " + iInvokedMethod.getTestMethod().getMethodName()); } @Override public void afterInvocation(IInvokedMethod iInvokedMethod, ITestResult iTestResult) { System.out.println("Successfully Tested : " + iInvokedMethod.getTestMethod().getMethodName()); } } 

Our testng.xml is defined as: -

 <!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" > <suite name="TestSuite" verbose="1" parallel="classes" thread-count="10"> <listeners> <listener class-name="core.TestExecutionListener"/> </listeners> <test name="IntegrationTests"> <classes> <class name="test.SomeTest1"/> <class name="test.SomeTest2"/> <class name="test.SomeTest3"/> <class name="test.SomeTest4"/> ... There are more than 20 classes </classes> </test> </suite> 

When we run the tests, the output we get is as follows:

 Testing : SomeTest1Method1 Testing : SomeTest2Method2 Testing : SomeTest4Method5 Successfully Tested : SomeTest2Method2 Successfully Tested : SomeTest4Method5 

while we expect the output to be: -

 Testing : SomeTest1Method1 Successfully Tested : SomeTest1Method1 Testing : SomeTest2Method2 Successfully Tested : SomeTest2Method2 Testing : SomeTest4Method5 Successfully Tested : SomeTest4Method5 

Guess this because of the parallel="classes" attribute in xml, since changing it to false gives the desired result. But, as is obvious, a change in execution requires much more time than parallel execution.

Is there a way to run these tests in parallel, but still get this output in sequence?

Answer 1

When you turn on the parallel, there will be some time in the logs between beforeInvocation and afterInvocation , as you noticed, and that the time difference changes from the test to check, therefore, the output is staggered.

If what you want is the start and end messages next to each other, you basically throw away the time factor and as such you can simply add the beforeInvocation message to the afterInvocation method as follows:

 public class TestExecutionListener implements IInvokedMethodListener { @Override public void beforeInvocation(IInvokedMethod iInvokedMethod, ITestResult iTestResult) { } @Override public void afterInvocation(IInvokedMethod iInvokedMethod, ITestResult iTestResult) { System.out.println("Testing : " + iInvokedMethod.getTestMethod().getMethodName()); System.out.println("Successfully Tested : " + iInvokedMethod.getTestMethod().getMethodName()); } } 

IMO is the only way to do this according to your specification. However, if there is other information that needs to be collected during the tests, perhaps you could buffer some of the logs in the TestExecutionListener , for example:

 public class TestExecutionListener implements IInvokedMethodListener { private Map<Integer, Deque<String>> logsMap = new HashMap<Integer, Deque<String>>(); public void log(IInvokedMethod iInvokedMethod, String log) { if(!logsMap.containsKey(iInvokedMethod.getId())) { logsMap.put(iInvokedMethod.getId(), new ArrayDeque<String>()); } logsMap.get(iInvokedMethod.getId()).add(log); } @Override public void beforeInvocation(IInvokedMethod iInvokedMethod, ITestResult iTestResult) { log(iInvokedMethod, "Testing : " + iInvokedMethod.getTestMethod().getMethodName()); } @Override public void afterInvocation(IInvokedMethod iInvokedMethod, ITestResult iTestResult) { log(iInvokedMethod, "Successfully Tested : " + iInvokedMethod.getTestMethod().getMethodName()); Deque<String> logs = logsMap.get(iInvokedMethod.getId()); while(!logs.isEmpty()) { System.out.println(logs.poll()); } } } 

Answer 2

First of all, I would recommend you use log4j or similar logging tools that are elegant, modular, and provide a lot of flexibility. It also paves the way for developing and processing a registration mechanism in several ways.

If you have a business requirement for which you need to track the logs of each stream separately, you should consider streaming these logs to different files. Log4j recording methods support the requirement of writing to multiple files. You can just do a Google search, you will get a lot of information.