How is connection testing with a third-party API suitable for continuous integration?

Question

I wrote a test some time ago that tests the integration that I wrote between my code and a third-party API. The test ensures that the integration works correctly and we will return the expected results.

The official build failed today because the test received 500 errors while trying to connect to a third-party API.

Does it make sense to test this situation?

Answer 1

In my opinion, integration tests are fine if a third-party (db, webservice, etc.) is not available. If you do not want to test the integration itself and just ordinary functions, you can make fun of the result of your API and test them. In this case, your test no longer depends on the availability of a third party.

I usually mark unit tests based on the availability of third-party vendors with a group attribute, such as Integration, and exclude them from the ongoing integration process. Instead, I let them work in the nightly build. Integration tests are usually time-consuming, and the point of continuous integration is to provide fast feedback .

Answer 2

No, this will not help your test suite fail if a third-party service is down. But you want to fully test your integration with the service. The following strategy worked well for me:

• Isolate a third-party service in a module (say, a class, but your language is modular nonetheless), which does as little as possible, in addition to encapsulating the connection to the service. Write methods in the class so that you can tell the difference between errors that are a service error and errors that are your error. For example, let's get an exception to the maintenance of a regular superclass.

• Write class tests of a class of service to

  • if a service error occurs, the tests pass but report an error.

  • if any other error occurs, as usual, terminates.

  Write these tests so that they work against live service. There should be a small number of these tests, so they should not create a problem for the test suite runtime.

• Enclose or deride a class of service from all other tests, including integration tests. (As a result of the "integration tests", I’m talking about tests that check all levels of your own code, and do not interact with a third-party service.)

  When you remove or mock a service class from integration tests, do not drown or mock the open API of the service class, but rather close or ridicule some lower level, possibly the internal method of the service class or library that you use to connect to the service . This prevents integration errors when calling a service class. In unit tests, drown or trick the public API of a service class, like any class.

As Martin Buberl suggests, it would be useful to conduct a separate integration test test in which the class of service would not be hatched or finished. Honestly, this is what was discussed in several projects in which I participated, but I do not think that this has ever been done. When a third-party service fails, we always learn about it from manufacturing errors (reported through monitoring or customer support) before we learn about it from the tests.