The Voidspace Techie Blog

Gravatar Arguably, that's just two sides of the same coin. It's also probably the result of separating the testing role from the programming role.

Perhaps tests are for finding out about what was wrong about what went wrong, while debugger are for finding what processes led to the error.

I would say that unless you have a particular kind of team, it's hard to use the spec to write your tests. Really, you need two kinds of test -- tests for bug-free code and tests for specification compliance.

2c...

-T


Gravatar You've completely missed the point. The testing role shouldn't be separated from the programmer role, tests aren't for finding out what went wrong because you write them before you write the code.

The tests are the spec! Yes you will also write regression tests though.


Gravatar I think we're talking past eachother. Perhaps I should have said that it's the result of inadvisedly separating the testing role from the programming role. I realise now that I was unclear.

In some situations of course, writing the tests first is simply impractical such as user-interface testing.

I certainly recognise the validity of the approach you're suggesting... But sometimes wringing a real specification out of the requirements just ain't that easy.

I think the principle of shipping thrown-together code out to the testing department is a pretty awful way to structure a coding project, but I still think it's nigh impossible to write tests based off the specifications based off the requirements that Jack wrote.... I've found that it's difficult to go to a non-programmer and get them to write the kind of tight requirements that I would need to be able to write software to meet them. The only tests I could write would be ones I wrote based off what I understood the goals to be...

But perhaps there is still something I am missing. Are there any good introductory articles to this topic?

-T


Gravatar It's absolutely true that well written tests articulate what the code should do and also prove that the code does indeed work. But, given that tests will fail when we inevitably break our code, surely a good debugger is a helpful tool to track down difficult to see bugs, especially with edge cases. We can then write new tests to cover these.


Gravatar "Perhaps I should have said that it's the result of inadvisedly separating the testing role from the programming role"

In which case you have worse problems anyway.

The whole point is that if you are doing TDD then the tests that *you* write (as a programmer) are a specification and not specifically for error checking.

"In some situations of course, writing the tests first is simply impractical such as user-interface testing."

I think this is just plain wrong. At Resolver we do functional testing for our user interface both GUI and through the web) using test first and we are far from alone in this.

"I still think it's nigh impossible to write tests based off the specifications based off the requirements that Jack wrote"

The programmer still needs to provide the specification for the code - obviously based on the feature you want to implement which is specified by the customer. In TDD writing the tests is part of the process of *creating the specification*. (No you won't be able to get your customer to write the specifications at the unit level!).

Kent Beck's book on extreme programming is the *best* introduction (and not too long). I'll have to see if I can hunt down some good examples though.


Gravatar Anthony: When tests break you have pretty much an exact pointerto where the failure is and what the conditions are. You can put your 'debugging information' into the failing test for further diagnosis.

As well as providing specifications, by decoupling and providing a test environment for individual units of your code your tests turn out to be fantastic debugging tools as well.

In two years at Resolver there have been less than a handful of occasions when we have found a debugger useful - ironically one of them is at the moment and it is actually a memory profiler we are using rather than a debugger per-se. We are tracking down some obscure leaks.


Gravatar Well, Giles comes off sounding like a pompous jerk. OK, that's probably not fair, but that's what I get from that one post.

Yes, I'm very interested in using tests as specifications (currently looking into NBehave and such), but it's not always that easy. His "It is my opinion that clean, well-tested code renders debuggers unnecessary, and that debuggers arose in the historical conditions which predated [T|B]DD" sounds like the Six Sigma people who say "if you properly control a production line, you don't need to test." Yeah, right, not in the real world. It also reminds me of the CMM people ("given perfect specifications and our process, we can guarentee bug free code", which doesn't address the elephant in the living room - specifications).

And, of course, real world systems (embedded, robotics) make life a lot harder. How can I use BDD/TDD to verify my system meets hard real time requirements (think braking system, engine control , etc)? How about a system that's controlling motors moving stuff around? And, most of the time, dealing with other people's code is evitable - like trying to port Linux, or the systems I work on, where a critical part of the code is real time code running on FPGA's (Verilog - nope, no Ruby there) and DSP's (again, no Ruby there).

And for a counter opinion, from here:
"Thinking back over my career, the time I've spent maintaining and fixing unit tests doesn't really pay for the relatively few bugs they've uncovered. In fact, when I have refactored code, many of the bugs are introduced by the unit tests themselves, not the source code."


Name:

Email:

URL:

Comment:  ? 

 

Commenting by HaloScan