Program written by generated code based on unit tests

Question

Since I was testing, I wondered if a hypothetical program could be fully developed by the generated test-based code. that is, it is possible to have a generator that creates code specifically for passing tests. Will the future of programming languages ​​just write tests?

Answer 1

I think this would not be easy, because at least for the first generations of such technology, developers would be very skeptical about the correctness of the code. Therefore, human visibility must also be involved.

As a simple illustration of what I mean, suppose you write 10 tests for a function, with input samples and expected results covering every scenario you can think of. A program can trivially generate code that passes all these tests, with something more than a rudimentary switch statement (your ten inputs correspond to their expected outputs). This code, obviously, would not be correct, but a person would need it.

This is just a simple example. It is easy to imagine more complex programs that cannot generate the switch statement, but still create solutions that are actually not correct, and that can be erroneous in much more subtle ways. Therefore, my proposal that any technology in this direction will be satisfied with a deep level of skepticism, at least at the beginning.

Answer 2

Although this is possible in the future, simple tests can be used to generate the code:

assertEquals(someclass.get_value(), true) 

but getting the right result from the black box integration test is what I think is NP-complete problem:

 assertEquals(someclass.do_something(1), file_content(/some/file)) assertEquals(someclass.do_something(2), file_content(/some/file)) assertEquals(someclass.do_something(2), file_content(/some/file2)) assertEquals(someclass.do_something(3), file_content(/some/file2)) 

Does this mean that the resulting code will always be written to / some / file? Does this mean that the resulting code should always be written to / some / file 2? Anyone can be true. What should I do if I need to run a minimal set to complete the tests? Without knowing the context and writing very precise and limiting tests, no code could determine (at the moment) what the author of the test intended.

Answer 3

If the code can be fully generated, then the basis of the generator should be a specification that accurately describes the code. This generator will then be a bit of a compiler that cross-compiles one language into another.

Tests are not such a language. They only claim that a particular aspect of the functionality of the code is valid and does not change. In doing so, they will raise the code so that it does not break, even if it is reorganized.

But how would you compare these two paths of development?

1) If the generator works correctly, the specification is always transmitted to the correct code. I claim that this code is tested by design and does not need additional testing. Better TDD generator than generated code.

2) Do you have a specification that leads to the generated code or specifications expressed in tests that ensure that the code works in my eyes.

3) You can combine both ways of development. Create a program structure with a proven generator from the specification, and then enrich the generated code with TDD. Attention: then you have two different development cycles in one project. This means that you must ensure that you can always restore the generated code when the specification changes, and that your additional code still fits correctly in the generated code.

Just one small example: Imagine a tool that can generate code from a UML class diagram. This can be done so that you can develop methods using TDD, but the class structure is defined in UML and you won’t need to test it again.