Be a little cautious when attributing errors to the first artifact in a chain of artifacts that lead to software.
The root cause of EVERY problem could as well be "bad specification" as "bad requirements" or "bad scope" or "bad team" or "bad leadership". They all contribute. Indeed, since the requirements are first, they can be pointed to any time you're in doubt of the cause.
I've worked on BIG projects and the secret sauce is to have one of those few people can write "requirements" that aren't just rubbish programming in English. There's an art form to saying: "the system shall do [X]" in a way that allows clever implementations, AND can be used by lawyers to sue when the software does not do "[X]".
Most things labeled requirements are a useless admixture of quality features and programming-in-English. They aren't the "proximate" cause of the problem, but they just happen to be in the causality chain.
Since software is so complex, we need more refined ways of pointing the finger. We need quality checks at each step so we can identify the failing step.
http://homepage.mac.com/s_lott/i...3218/
index.html
See 'Can Technology Eliminate Human Error' for at least one independent study on the subject. http://www.4-
sightconsulting.co....uman_error.html
I understand what you are saying, on a basic level if the specification doesn't say 'there should be no bugs'; then any bug is theoretically a failure in specification !
But even so 'specifying software is difficult', and mistakes in the first step will percolate through. 'A chain is only as strong as the weakest link'.
There has to be meaningful analysis that can be done on software specification, and its relationship to software errors, and a lot of people in the engineering industry are very worried about this.
A few notes:
1) I think you'd be hardpressed to find a proper software engineer (or any proper engineer) who expects ALL the requirements to be explicitly stated up front. Requirements specification is often an ongoing process. The only times that you can expect to do full requirements specification at the beginning (ie. waterfall model) is on small projects, or projects where the problem domain is very well known and understood by all involved.
2) I myself love Agile techniques, but you have to remember that testing is only one way to verify a system. And it definitely shouldn't be the only way. If someone was implementing the shutdown sequence for a nuclear power plant, would you be happy with a whole bunch of unit/functional/regression tests, or would you expect something a bit more?
Not a lot of people know it, but you can use automated theorem provers to mathematically prove the correctness of your specifications, both for consistency and completeness.
Mathematically specified requirements are *key* for any safety critical application, as human language is too ambiguous and imprecise. There are often multiple interpretations of a statement, even if you think you wrote it such that only one is possible.
This is exactly what was done for the Darlington Nuclear Generation Station. A nice set of slides showing the formalization of some of their requirements is here: http://www.cas.mcmaster.ca/~lawf...tes/
wassyng.pdf (and there are tons of other papers available in the typical locations, my profs wrote a lot about their experiences working on Darlington).
They would build their requirements in an iterative process, but the end product was always specification via tables, in a purely mathematical form (with English annotations to help initial understanding). These mathematical forms were then automatically checked by systems like PVS (http://pvs.csl.sri.com/) for consistency and completeness.
And I know for a fact that PVS found issues with their requirements that no human involved in the process noticed.
Thanks Jay, interesting stuff. I'll forward your comment to my Dad.
Commenting by HaloScan
