The design process for digital ASICs spends approximately 70% of resources on verification, as making a duff chip is very expensive. What we do is spend time nailing down the spec, then have a design team designing to the spec as well as a verification team testing from the spec. The verification tewam will create their own implementation of the spec in a high level verification language and use it to either generate test sequences for the designers efforts and/or check the results from the design during simulation.
We end up with a self checking testbench and an easy way to create test scenarios.
Random testing: The Hardware Verification Languages (Specman-e, SystemVerilog, Vera, SCV, testbuilder), have the ability not only to generate random sequences, but also to solve and generate random number streams subject to many constraints - for example, generate packets of size xwhere 4
Missing part of the comment by Paddy:
... of size x, where 4
where 4 <=x<=16 with values 0<= v<256.
Later on you can modify a test by just adding constraints, e.g. `if 6<=x<8: 0<=v<16` will generate a modified stream of packets to create a new test .
Random testing, when done right has the advantage of exploring the state space of the design in a steerable, but not fully predictable manner leading to the discovery of non-obvious bugs.
Coverage: HVL,s can monitor the internal state variables of the design and so extract coverage data for state machines and variables. Tests can be written to be coverage sensitive.
By creating and testing two independent models againt each other, you get the benefit of two interpretations of the spec and a self checking testbench. By monitoring bug rates over time as well as design coverage, You get a feel for when testing is` done' (or some figures behind good enough).
To map that to high reliability Software development then the software to ship would be the equivalent of the designers model. It should be written in what is an appropriate language for the environment - C, C++, Forth, Java, C#, ...
The Verification team should be using a language optimesed for Verification and modelling. They could gain inspiration from the set of hardware verification languages given previously, but they would gain by being able to 'couple' to the internal state of the design language for measuring coverage.
I'm probably `teaching your father to suck eggs` as the saying goes but I would be interested in his views as I find little evidence of constrained random testing of software outside the Electronic Design Automation field.
- Paddy.
Commenting by HaloScan
