Best Practices for TDD and Reporting

Question

I am trying to become more familiar with test-based approaches. One of the drawbacks for me is that the bulk of my code generates context for reporting (PDF documents, graphic images). There is always a complex designer, and there is no simple test for correctness. There is no way to check only fragments!

Do you know TDD methods for this situation?

Answer 1

You can use acceptance test technology to replace unit tests and have verified reports for well-known data used as references.

However, this type of test does not provide a fine-grained diagnosis, as unit tests do, they usually only provide a PASS / FAIL result, and if reports change frequently, links need to be regenerated and re-checked as Well.

Answer 2

Some applications or frameworks simply inherit the unit test independently, and you really can't do much.

I prefer to avoid such frameworks altogether, but if it is absolutely necessary to solve such problems, it may be useful to extract all the logic into the tested library, leaving only declarative code in the framework.

Answer 3

The question I ask myself in these situations is " how did I know that I got it right "

I wrote a lot of code in my career, and almost all of this didn't work the first time. Almost every time I go back and change the code to refactor, change functions, performance or fix bugs, I broke it again. TDD protects me from me (thank god!).

In the case of the generated code, I do not feel obligated to check the code. That is, I trust the code generator. However, I want to check my inputs on code generators. Exactly how to do this, it depends on the situation, but the general approach is to ask yourself how I can make a mistake, and then figure out how to verify that I understood correctly.

Maybe I'm writing an automated test. Maybe I will check something manually, but this is pretty risky. Maybe something else. It depends on situation.

Answer 4

You can try to use the web service to source the report data and verify this, but you will not have unit tests for rendering. This is the same problem you experience when testing your views. Sure, you can use web testing like Selenium, but you probably won't practice true TDD. You will create tests after your code is executed.

In short, use common sense. There is probably no point in trying to test the rendering of the report. You may have manual test cases that the tester will have to go through manually or just check the reports yourself.

You can also check " How much Unit Test Coverage do you need? - Testivus Answer "

Answer 5

To twist the answers from Mark Seeman and Jay Bazuzi a little:

Your problem is that the front part of the reports creates a data format, the output of which you cannot easily check in the “check” part of your tests.

The way to solve this problem is as follows:

• Have some very high-level integration tests that superficially verify that your internal code translates correctly into your interface code. I usually call these tests "smoke tests", as in "if I turn on the power and she smokes, it's bad."

• Find another way to test your reporting code. Either check the intermediate structure of the output data, or implement an alternative output interface that is more convenient for testing, HTML, plain text, whatever.

This seems like a common problem testing web applications: it is not possible to automatically verify that the "page looks right." But it’s enough to verify that the words and numbers in the page data are correct (using a software browser as a mechanization and page scraper) and have several surface functional tests (with Selenium or Windmill) if the page is critically dependent on Javascript.

Answer 6

Consider extracting text from a PDF and checking it. However, this will not give you formatting. Some PDF extractors can extract images if the graphics are in pdf format.

Answer 7

Faced with this situation, I try two approaches.

• The Golden Master approach. Create a report once, check it yourself, then save it as a "golden master". Write an automated test to compare its output with the gold master and fail if they are different.

• Automate tests for data, but check the format manually. I automate the verification of the module that generates the report data, but to check the format of the report, I generate a report with hard-coded values ​​and check the report manually.

I highly recommend that you do not generate a full report to verify that the data in the report is correct. When you want to check a report (not data), then generate a report; when you want to check the data (not the format) and then only generate the data.