When do BDD scripts get too specific?

Question

After a very inspiring training in TDD and BDD, I am trying to implement a methodology using MSTest and Specflow . But I have a question, what am I attached to:

I wrote Acceptance Tests to test the subsystem we are working on. Our system is a bit distributed:

• there is a third-party computer
• with its own launch freely
• with the third database, which we access through tcp / ip

However, the Specflow script seems too specialized for my own development setup : it contains inputs that are valid only for me. In the example below, the ip address is mostly accessible from me. And the target directory is the directory on my machine.

An accredited tester / validator or product owner will probably not be able to run the same test script because they will not have access to this machine. But my fellow developers cannot either.

 @lastOne Scenario: Get lattest 3rdParty OCR Data into specified directory Given I indicate 'database' as the databaseName of third party computer And I indicate '12.126.42.21' as the ipAddress of the third party computer And I indicate 'user' as the databaseUser in third party computer And I indicate 'c:\Temp\test_ocr\' as the destination path where to put the ocr data And I indicate '2013020800009E' as the truck identifier to be associated with ocr data When I call the OCR Application program Then the destination path should contain correctly named xml file, with validated xml data, and jpg files about ocr data. 

I am afraid that I have some misconceptions regarding BDD. Am I too specific in my scenario? If so, where to stay?

Answer 1

I'm not sure your question is BDD specific, but it's still good.

I usually recommend that all development be done with a continuous integration server that runs your tests every time you test, even for a private project where you work alone. Even my personal projects get this, because TeamCity is free, and the desktop for children at home is inactive when I register. Importance, if it is more obvious, if you work in a team because it ceases to be ever doubting when you get which will be built.

But it also stops your problem. You can tell very quickly when something is too specific, because it does not work on both your personal machine and the build machine. These problems exist, regardless of whether you work in BDD, TDD, ATDD or in any kind of testing.

Looking at your example above, I would say that it is very specific and also very fragile. If one of the third-party PCs shuts down one day, all your tests fail. If you used Specflow to run unit tests, I would recommend making fun of most of your code so you can test without invoking the test computer, but your example reads more as you try to perform system / integration testing.

Instead of specifying all your options separately, why not name the whole package

 Given Im using the test pc 

you can set many of them in a binding, and if you need to adapt them then, so that the tests pass anyway

 [Given] public void GivenImUsingTheTestPc() { if (Environment.ComputerName == "d1234") { ipadress = 1.2.3.4; .... 

This, obviously, only drives fragility, but at least it holds you now.