<forwarding email from Swapnil Dwivedi> -------- Original Message -------- Subject: RE: [sv-ec] New Mantis on Coverage Calculation Corner Cases Date: Fri, 10 Nov 2006 15:21:43 -0000 From: "Dwivedi, Swapnil" <swapnil_dwivedi@mentor.com> To: "Mark Strickland \(mastrick\)" <mastrick@cisco.com>, "Scott, David" <david_scott@mentor.com>, <sv-ec@server.eda.org> Cc: "Arturo Salz" <Arturo.Salz@synopsys.com>, "Warmke, Doug" <doug_warmke@mentor.com> Hello Mark! I surely do think you have a valid requirement. Users would definitively like to have functionality which you described in your email. But I think this feature should be part of the Coverage report scheme and onus should lie with vendor to provide it as part of their tool's report query system. The "weight" option has a special meaning in LRM and is currently solely attached to Coverage calculation. Inferring something else from it (at different situations) would possibly have different dictations from not only customers but vendors too. By keeping it in the confines of Coverage calculation we can minimize that. Now about your requirement of not showing in report the items with "weight == 0". I would forward your requirement to our team. If you do not mind we would be very much interested in knowing your other requirements and would directly like to discuss with you. In our product, "Questa", we do provide something similar in our Coverage reporting system : "[-above <coverage_percent> | -below <coverage_percent>]". You may use that for the moment until we come back about your requirement. Lets discuss your scenarios and how do we achieve your requirements. Scenario 1) No problem here. Scenario 2) No problem here either. Scenario 3) Covergroup A A.type_option.weight = 1 (default) A.type_option.goal = 100 Achieved = 100 Covergroup B : No Coverpoints (Say Covergroup under construction) B.type_option.weight = 1 (default) B.type_option.goal = 100 Achieved = 0 (no coverage objects/values => 0 achieved) Let assume one instance of each Covergroup. Clearly coverage would be Design: 50% (Covergroup A contribution: 100% Covergroup B contribution: 0%) Covergroup A coverage: 100% Covergroup B coverage: 0% Above confirms to one of your requirement. Now lets see what happens if you make the "B.type_option.weight = 0" and given my suggestion is used. Since weight is zero the concerned object does not contribute in Coverage calculation and the new Coverage is Design: 100% (Covergroup A contribution: 100% Covergroup B: None) Covergroup A coverage: 100% Covergroup B coverage: 0% This one almost confirms to your other requirement. You expect "Covergroup B coverage: 100%" but it is "0%". As a user working in group I think the results shown is perfect. When you are working on a new Covergroup your first aim is to see the you do not un-intentionally affect the Design Coverage (or get an ire from colleague) by suddenly bringing down the Coverage. Your second aim is to keep seeing your Covergroup's correct coverage so that you can keep an eye on it and keep working on it. Of course there are other uses of "weight = 0". For example you do not want another group's Covergroup to affect your Coverage objectives/goals. You can create a file with hierarchical assignments of "weight = 0" and mask their Covergroup out of coverage calculation. Scenario 4) Covergroup A A.type_option.weight = 1 (default) A.type_option.goal = 100 Achieved = 100 Covergroup B : One coverpoint "cvp" B.type_option.weight = 1 (default) B.type_option.goal = 100 Achieved = 0 (cvp did not contribute) B.cvp.type_option.weight = 0 (i.e. does not contribute in coverage calculation) B.cvp.goal = 100 Achieved = 50 So the coverage is Design: 50% (Covergroup A contribution: 100% Covergroup B contribution: 0%) Covergroup A coverage: 100% Covergroup B coverage: 0% (cvp contribution : None) Coverpoint B.cvp coverage: 50% Now if I make "B.type_option.weight = 0" and "B.cvp.type_option.weight = 0" then (lets call this CASE A) Design: 100% (Covergroup A contribution: 100% Covergroup B contribution: None) Covergroup A coverage: 100% Covergroup B coverage: 0% (cvp contribution : None) Coverpoint B.cvp coverage: 50% Now if I make "B.type_option.weight = 0" and "B.cvp.type_option.weight = 1 (default)" (lets call this CASE B) Design: 100% (Covergroup A contribution: 100% Covergroup B contribution: None) Covergroup A coverage: 100% Covergroup B coverage: 50% Coverpoint B.cvp coverage: 50% Now as a user "CASE B" fits perfectly when working in group/multi-group environment. Scenario 5) This is same as CASE A of the scenario 4. Covergroup A A.type_option.weight = 1 (default) A.type_option.goal = 100 Achieved = 100 Covergroup B : One coverpoint "cvp" B.type_option.weight = 0 B.type_option.goal = 100 Achieved = 0 (both cvp and B do not contribute) B.cvp.type_option.weight = 0 (i.e. does not contribute in coverage calculation) B.cvp.goal = 100 Achieved = 50 Design: 100% (Covergroup A contribution: 100% Covergroup B contribution: None) Covergroup A coverage: 100% Covergroup B coverage: 0% Coverpoint B.cvp coverage: 50% As you can see "CASE B (scenario 4)" is very reasonable for almost all the occasions. As to various other flavors which we did not discuss, the vendor should be responsible for providing it as part of their reporting structure. Q) > On a somewhat different topic, is there a concept of a choice > between hierarchical grading and flat grading provided to the > user? To illustrate the difference, assume a covergroup with > 100% grade with just one bin included and another covergroup > with 0% grade and 99 bins included. Assuming the groups have > the same weight, the hierarchical grading scheme would give an > overall grade of 50%, while flat grading would give 1%. One > way to let the user specify may be to merge groups that all > have default weight as if their weight was equal to their > number of bins while merging explicitly weighted groups in a > hierarchical manner. A) If I understand your requirement correctly then answer is no. Covergroup coverage calculation is kind of based on name. Merging of counts/coverage data is done with name of the object as the key. With two different Covergroups, associating names across Covergroups etc would soon make things go out of control. But interestingly something similar is there within a Covergroup. Given your example above, B.cvp.get_coverage() actually aggregates the bins at coverpoint level with name of the bin as the key. Same is true for Cross. So if your intention is to have two distinct sets of bins but coverage calculation to be done on them in an aggregate fashion then use Covergroup with ref arguments and parameters. For example covergroup cvg (ref cvp_expr, input int bin_size, int left_bound, int right_bound) { coverpoint cvp_expr { bins user_bin [bin_size] = {[left_cound:right_bound]}; } } cvg cvg_instance1 = new(a, 10, 1, 100); cvg cvg_instance1 = new(b, 5, 10, 25); initial $display("Aggregate coverage = %f", cvg.cvp.get_coverage()); If you have further questions/queries then please feel free to send an email to us at support@model.com Regards, Swapnil ++ Swapnil Dwivedi, Sr. Software Engr. Mentor Graphics, San Jose, CA Office : +44 (1635) 811-442 (Internal: 8210-1442) Cell : +44 (7919) 150-796 Home : +1 (408) 259-1817 (VOIP) > -----Original Message----- > From: Mark Strickland (mastrick) [mailto:mastrick@cisco.com] > Sent: Thursday, November 09, 2006 9:43 PM > To: Dwivedi, Swapnil; Scott, David; sv-ec@eda.org > Cc: Arturo Salz; Warmke, Doug > Subject: RE: [sv-ec] New Mantis on Coverage Calculation Corner Cases > > Swapnil - You could certainly define "weight=0" to mean "I > don't want to see this in the report" and make the algorithm > simple. However, that eliminates a useful feature of "I want > to see this in the report but I don't want it to affect the > overall calculation". One place I find this useful is when I > have a cross of two coverpoints, and I want to see the > results for the points themselves (for understanding what is > needed for coverage closure), but not have them contribute > twice (once themselves and once in the cross) to the overall result. > > Dave - I agree the rule must apply at all levels and that > user-specified values should not be overwritten. The > calculation could use a modified version of what the weight > field shows. However, I am a user, not a tool creator, so I > think I should just state the desired user view. > > Example: > Lets say my design has two covergroups, A and B. A has a > weight of 1 (by the way, that is the default if I don't say > anything, right?) and a grade of 100%. B varies by scenario > shown below: > > Scenario 1) B has a weight of 1 and one coverpoint with a > grade of 50% and a non-zero weight. The overall grade for > this design is 75% and the grade of B is 50%. > > Scenario 2) B has a weight of 0 and one coverpoint with a > grade of 50% and a non-zero weight. The overall grade is > 100% and the grade of B is 50%. > > Scenario 3) B has a weight of 1 and no coverpoints. This is > a tricky one. Most of the time, I would prefer the overall > grade and the grade of B to be 100%, because I do not have > any coverage holes. However, occasionally it would be nice > to see the overall grade reflect the fact that there is a > covergroup I am intending to populate but have not yet done > so. In this case, the overall grade would be 50% and the > grade of B would be 0%. How can both modes work? > Option a) Always show the latter grades and make the user > explicitly exclude B if he/she wanted the former. The user > may spend time debugging and making this fix. > Option b) If the weight for B was unassigned in user code > (i.e. default value), go with the first mode. If it was > assigned (even if it was assigned to 1), go with the second. > > Scenario 4) B has a weight of 1 and one coverpoint with a > grade of 50% and a weight of 0. If we go with option (a) for > scenario (3), then the consistent behavior would be to have > the overall grade be 50%. If B has a grade of 50%, we are > obscuring the fact that it contributed 0% to the calculation > for the overall grade. If B has a grade of 0%, we are > obscuring the fact that there was some coverage achievement > in this group albeit with zero weight. If we go with option > (b) for scenario (3), then the consistent behavior is to have > the overall grade be 100% if B had an unassigned weight or > 50% if it was assigned to 1. > > Scenario 5) B has a weight of 0 and one coverpoint with a > grade of 50% and a weight of 0. The overall grade is 100% > and the grade of B is 50%. > (This scenario is very like scenario 2; there is no reason > that assigning the 0 weight to the coverpoint should obscure > its grade as viewed at the level of B.) > > In all cases of scenarios 2 through 5 in which there was a > grade that did not contribute to the overall grade, it would > be ideal if some flag was set so that a GUI could highlight that fact. > > On a somewhat different topic, is there a concept of a choice > between hierarchical grading and flat grading provided to the > user? To illustrate the difference, assume a covergroup with > 100% grade with just one bin included and another covergroup > with 0% grade and 99 bins included. Assuming the groups have > the same weight, the hiearchical grading scheme would give an > overall grade of 50%, while flat grading would give 1%. One > way to let the user specify may be to merge groups that all > have default weight as if their weight was equal to their > number of bins while merging explicitly weighted groups in a > hierarchical manner. > > Mark > > -----Original Message----- > From: Dwivedi, Swapnil [mailto:swapnil_dwivedi@mentor.com] > Sent: Thursday, November 09, 2006 8:12 AM > To: Scott, David; Mark Strickland (mastrick); sv-ec@eda.org > Cc: Arturo Salz; Warmke, Doug > Subject: RE: [sv-ec] New Mantis on Coverage Calculation Corner Cases > > I have a suggestion. For the case of weight being zero we can > take hint from Coverpoint bins. > > If a Coverpoint bin, after applying the ignore and illegal > values, ends up with no values (they are called empty bins) > then we take the bin out of the coverage calculation. > Continuing further if all the bins in a Coverpoint are empty > (or there are no bins in Coverpoint) then effectively > Coverpoint does not have any bins and it is taken out of > Coverage calculation. And so on. > > The point that user is giving a zero means he/she is not > bothered or concerned by the contribution of the object > (Covergroup/Coverpoint/Cross). We can apply the same rule as > above. A zero weight means just take the object out of > Coverage calculation. So if the all the Coverpoints and > Crosses are of zero weight then they do not exist and hence > Covergroup object does not (i.e. it might exist as instance > but does not contribute in the Coverage calculation and a > smart simulator might as well purge it for all purpose). > > Regards, > Swapnil > > > > > -----Original Message----- > > From: owner-sv-ec@server.eda.org > > [mailto:owner-sv-ec@server.eda.org] On Behalf Of Scott, David > > Sent: Thursday, November 09, 2006 2:07 AM > > To: Mark Strickland (mastrick); sv-ec@server.eda.org > > Cc: Arturo Salz; Warmke, Doug > > Subject: Re: [sv-ec] New Mantis on Coverage Calculation Corner Cases > > > > I had a reaction to this one that I posted internally here > at Mentor > > last week; I've been encouraged to post it to SV-EC ... > > > > > > If I read Mark's suggestion right, I think that is saying that > > all-zero weights propagate as zero weight upward, i.e., all > > coverpoints/crosses weighted zero imply covergroup weight = > 0. Is that > > > right? > > > > I'm worried about a couple of things. > > > > One is that while this works out OK for the coverpoint/cross to > > covergroup level, presumably there is also weighting among > covergroups > > > to form the $get_coverage() value. So now you have to answer the > > question what $get_coverage() returns if all covergroup types are > > weighted 0. Mark's argument can't apply at this level, I > don't think. > > I'm a little bothered by an algorithm not general enough to > apply at > > all levels. > > > > A more serious issue, I think, is this: > > > > covergroup ct; > > type_option.weight = 1; > > coverpoint i { type_option.weight = 0; } coverpoint j { > > type_option.weight = 0; } endgroup ct cv = new; > > > > initial $display(ct::type_option.weight); > > > > If I have interpreted Mark's suggestion correctly, this > will have to > > display 0, in contradiction of the explicit assignment. > > > > -- Dave Scott > > > > > > Mark Strickland (mastrick) wrote: > > > Doug/Arturo, > > > > > > I agree that negative weights are not needed and could be > > considered an > > > error. However, I'm not sure about zero coverage when all > > weights are > > > zero. I would like to be able to have two possibilities > > for a coverage > > > point I did not want to participate in the overall coverage > > grade: I can > > > still see its grade or I don't see it at all. The weight > > of zero is how > > > I would achieve the first. If a group of coverage points > > who all have > > > weight of zero show me a grade of zero, then I am prevented > > from seeing > > > the actual achievement for that group. The solution that > > allows me to > > > see the grade for a group with all weights zero but does > > not affect the > > > overall grade is to calculate the grade as if all the > > weights were one > > > but then assign a weight of zero to the group grade. > > > > > > Mark > > > > > > -----Original Message----- > > > From: owner-sv-ec@eda.org [mailto:owner-sv-ec@eda.org] On > Behalf Of > > > Arturo Salz > > > Sent: Monday, October 30, 2006 2:45 PM > > > To: Warmke, Doug; sv-ec@eda.org > > > Subject: RE: [sv-ec] New Mantis on Coverage Calculation > Corner Cases > > > > > > Doug, > > > > > > I added a bugnote expressing my opinion: > > > > > > 1) If the sum of all weights (Wi) is zero then the coverage > > (Cg) should > > > yield zero. > > > > > > 2) A negative weight should be an error. Unsigned weights > > are likely to > > > just generate garbage reports and force more work upon users. > > > > > > Arturo > > > > > > -----Original Message----- > > > From: owner-sv-ec@eda.org [mailto:owner-sv-ec@eda.org] On > Behalf Of > > > Warmke, Doug > > > Sent: Saturday, October 28, 2006 4:22 PM > > > To: sv-ec@eda.org > > > Subject: [sv-ec] New Mantis on Coverage Calculation Corner Cases > > > > > > Hi All, > > > > > > Some simple corner cases were identified in 18.10 on Coverage > > > Calculations. The first is if the sum of all weights Wi in the > > > denominator of the Cg equation is 0. > > > The second is what to do about negative weights. > > > > > > Can you please read > > > http://eda.org/svdb/bug_view_page.php?bug_id=0001655 > > > and express any opinions on the two corner cases? > > > > > > Thanks, > > > Doug > > > > > > > > > > >Received on Fri Nov 10 13:43:48 2006
This archive was generated by hypermail 2.1.8 : Fri Nov 10 2006 - 13:44:12 PST