RE: [sv-ec] New Mantis on Coverage Calculation Corner Cases

Swapnil - You could certainly define "weight=0" to mean "I don't want to
see this in the report" and make the algorithm simple.  However, that
eliminates a useful feature of "I want to see this in the report but I
don't want it to affect the overall calculation".  One place I find this
useful is when I have a cross of two coverpoints, and I want to see the
results for the points themselves (for understanding what is needed for
coverage closure), but not have them contribute twice (once themselves
and once in the cross) to the overall result.

Dave - I agree the rule must apply at all levels and that user-specified
values should not be overwritten.  The calculation could use a modified
version of what the weight field shows.  However, I am a user, not a
tool creator, so I think I should just state the desired user view.

Example:
Lets say my design has two covergroups, A and B.  A has a weight of 1
(by the way, that is the default if I don't say anything, right?) and a
grade of 100%.  B varies by scenario shown below:

Scenario 1) B has a weight of 1 and one coverpoint with a grade of 50%
and a non-zero weight.  The overall grade for this design is 75% and the
grade of B is 50%.

Scenario 2) B has a weight of 0 and one coverpoint with a grade of 50%
and a non-zero weight.  The overall grade is 100% and the grade of B is
50%.

Scenario 3) B has a weight of 1 and no coverpoints.  This is a tricky
one.  Most of the time, I would prefer the overall grade and the grade
of B to be 100%, because I do not have any coverage holes.  However,
occasionally it would be nice to see the overall grade reflect the fact
that there is a covergroup I am intending to populate but have not yet
done so.  In this case, the overall grade would be 50% and the grade of
B would be 0%.  How can both modes work?  
Option a) Always show the latter grades and make the user explicitly
exclude B if he/she wanted the former.  The user may spend time
debugging and making this fix.
Option b) If the weight for B was unassigned in user code (i.e. default
value), go with the first mode.  If it was assigned (even if it was
assigned to 1), go with the second.  

Scenario 4) B has a weight of 1 and one coverpoint with a grade of 50%
and a weight of 0.  If we go with option (a) for scenario (3), then the
consistent behavior would be to have the overall grade be 50%.  If B has
a grade of 50%, we are obscuring the fact that it contributed 0% to the
calculation for the overall grade.  If B has a grade of 0%, we are
obscuring the fact that there was some coverage achievement in this
group albeit with zero weight.  If we go with option (b) for scenario
(3), then the consistent behavior is to have the overall grade be 100%
if B had an unassigned weight or 50% if it was assigned to 1.

Scenario 5) B has a weight of 0 and one coverpoint with a grade of 50%
and a weight of 0.  The overall grade is 100% and the grade of B is 50%.
(This scenario is very like scenario 2; there is no reason that
assigning the 0 weight to the coverpoint should obscure its grade as
viewed at the level of B.)

In all cases of scenarios 2 through 5 in which there was a grade that
did not contribute to the overall grade, it would be ideal if some flag
was set so that a GUI could highlight that fact.

On a somewhat different topic, is there a concept of a choice between
hierarchical grading and flat grading provided to the user?  To
illustrate the difference, assume a covergroup with 100% grade with just
one bin included and another covergroup with 0% grade and 99 bins
included.  Assuming the groups have the same weight, the hiearchical
grading scheme would give an overall grade of 50%, while flat grading
would give 1%.  One way to let the user specify may be to merge groups
that all have default weight as if their weight was equal to their
number of bins while merging explicitly weighted groups in a
hierarchical manner.

Mark

-----Original Message-----
From: Dwivedi, Swapnil [mailto:swapnil_dwivedi@mentor.com] 
Sent: Thursday, November 09, 2006 8:12 AM
To: Scott, David; Mark Strickland (mastrick); sv-ec@eda.org
Cc: Arturo Salz; Warmke, Doug
Subject: RE: [sv-ec] New Mantis on Coverage Calculation Corner Cases

I have a suggestion. For the case of weight being zero we can take hint
from Coverpoint bins.

If a Coverpoint bin, after applying the ignore and illegal values, ends
up with no values (they are called empty bins) then we take the bin out
of the coverage calculation. Continuing further if all the bins in a
Coverpoint are empty (or there are no bins in Coverpoint) then
effectively Coverpoint does not have any bins and it is taken out of
Coverage calculation. And so on.

The point that user is giving a zero means he/she is not bothered or
concerned by the contribution of the object
(Covergroup/Coverpoint/Cross). We can apply the same rule as above. A
zero weight means just take the object out of Coverage calculation. So
if the all the Coverpoints and Crosses are of zero weight then they do
not exist and hence Covergroup object does not (i.e. it might exist as
instance but does not contribute in the Coverage calculation and a smart
simulator might as well purge it for all purpose).

Regards,
Swapnil



> -----Original Message-----
> From: owner-sv-ec@server.eda.org
> [mailto:owner-sv-ec@server.eda.org] On Behalf Of Scott, David
> Sent: Thursday, November 09, 2006 2:07 AM
> To: Mark Strickland (mastrick); sv-ec@server.eda.org
> Cc: Arturo Salz; Warmke, Doug
> Subject: Re: [sv-ec] New Mantis on Coverage Calculation Corner Cases
> 
> I had a reaction to this one that I posted internally here at Mentor 
> last week; I've been encouraged to post it to SV-EC ...
> 
> 
> If I read Mark's suggestion right, I think that is saying that 
> all-zero weights propagate as zero weight upward, i.e., all 
> coverpoints/crosses weighted zero imply covergroup weight = 0. Is that

> right?
> 
> I'm worried about a couple of things.
> 
> One is that while this works out OK for the coverpoint/cross to 
> covergroup level, presumably there is also weighting among covergroups

> to form the $get_coverage() value. So now you have to answer the 
> question what $get_coverage() returns if all covergroup types are 
> weighted 0. Mark's argument can't apply at this level, I don't think.
> I'm a little bothered by an algorithm not general enough to apply at 
> all levels.
> 
> A more serious issue, I think, is this:
> 
> covergroup ct;
> type_option.weight = 1;
> coverpoint i { type_option.weight = 0; } coverpoint j { 
> type_option.weight = 0; } endgroup ct cv = new;
> 
> initial $display(ct::type_option.weight);
> 
> If I have interpreted Mark's suggestion correctly, this will have to 
> display 0, in contradiction of the explicit assignment.
> 
> -- Dave Scott
> 
> 
> Mark Strickland (mastrick) wrote:
> > Doug/Arturo,
> >
> > I agree that negative weights are not needed and could be
> considered an
> > error.  However, I'm not sure about zero coverage when all
> weights are
> > zero.  I would like to be able to have two possibilities
> for a coverage
> > point I did not want to participate in the overall coverage
> grade: I can
> > still see its grade or I don't see it at all.  The weight
> of zero is how
> > I would achieve the first.  If a group of coverage points
> who all have
> > weight of zero show me a grade of zero, then I am prevented
> from seeing
> > the actual achievement for that group.  The solution that
> allows me to
> > see the grade for a group with all weights zero but does
> not affect the
> > overall grade is to calculate the grade as if all the
> weights were one
> > but then assign a weight of zero to the group grade.
> >
> > Mark
> >
> > -----Original Message-----
> > From: owner-sv-ec@eda.org [mailto:owner-sv-ec@eda.org] On Behalf Of 
> > Arturo Salz
> > Sent: Monday, October 30, 2006 2:45 PM
> > To: Warmke, Doug; sv-ec@eda.org
> > Subject: RE: [sv-ec] New Mantis on Coverage Calculation Corner Cases
> >
> > Doug,
> >
> > I added a bugnote expressing my opinion:
> >
> > 1) If the sum of all weights (Wi) is zero then the coverage
> (Cg) should
> > yield zero.
> >
> > 2) A negative weight should be an error. Unsigned weights
> are likely to
> > just generate garbage reports and force more work upon users.
> >
> > 	Arturo
> >
> > -----Original Message-----
> > From: owner-sv-ec@eda.org [mailto:owner-sv-ec@eda.org] On Behalf Of 
> > Warmke, Doug
> > Sent: Saturday, October 28, 2006 4:22 PM
> > To: sv-ec@eda.org
> > Subject: [sv-ec] New Mantis on Coverage Calculation Corner Cases
> >
> > Hi All,
> >
> > Some simple corner cases were identified in 18.10 on Coverage 
> > Calculations.  The first is if the sum of all weights Wi in the 
> > denominator of the Cg equation is 0.
> > The second is what to do about negative weights.
> >
> > Can you please read
> >   http://eda.org/svdb/bug_view_page.php?bug_id=0001655
> > and express any opinions on the two corner cases?
> >
> > Thanks,
> > Doug
> >
> >   
> 
>