Hi Swapnajit, So I believe we are largely in agreement. As for the schedule. First, the committee must vote on this issue. Once it's decided, we can add it to the list of issues to be published as a corrigendum (for the details see http://standards.ieee.org/faqs/Amendments_Corrigenda.ppt ). Now that the 1800-WG has voted on how to proceed next, perhaps we can discuss our options in the technical committee. Arturo ________________________________ From: Swapnajit Chakraborti [mailto:swapnaj@cadence.com] Sent: Friday, January 20, 2006 6:15 AM To: Arturo Salz; Gordon Vreugdenhil Cc: sv-ec@eda.org Subject: RE: [sv-ec] Completing my coverage thought... Hi Arturo, Thanks for your elaborate explanation. Here are my inputs on this topic: - "empty" bins should be eliminated from coverage calculation ( from both numerator and denominator). Note that we need to remove them from denominator while calculating covered percentage and we need to remove them from numerator while calculating uncovered precentage. Such bins need not be reported as well by any reporting tool as uncovered. One argument for this can be that if user has specified some bins as "ignore_bins", there is no point in reporting them as uncovered. - For the corner case i.e. "empty" coverpoint, option (4) looks better than the other three. But in that case, reporting tools will have to decide how to represent this to the user. Having said that, my question is when are we going to freeze this new semantic? Thx, Swapnajit. ________________________________ From: Arturo Salz [mailto:Arturo.Salz@synopsys.com] Sent: Thursday, January 19, 2006 2:45 AM To: Gordon Vreugdenhil; Swapnajit Chakraborti Cc: sv-ec@eda.org Subject: RE: [sv-ec] Completing my coverage thought... I'll try to clarify. In my previous response to Swapnajit I was trying to make it clear that ignore bins do not affect the "overall coverage space". What I meant (and I believe that was the gist of the question) is that an "ignore bins" specification does not affect how bins are computed. Otherwise, one might consider removing the ignored values from the space and then distribute the remaining (non ignored) values into the resulting bins. Until the recent discussion, what we now call "empty bins" was not considered special in any way (other than the fact that it could never be covered). We are now refining our thinking to remove the contribution to coverage of these empty bins. This new thinking does indeed change the way coverage is computed from the way I had described it in my earlier response to Swapnajit. Our latest thinking is that empty bins should not affect coverage. Gord has identified one corner case (a coverpoint with all empty bins) and proposed a solution, but I don't believe we have consensus on how to deal with it. My thinking is that there are four alternatives (the first three were proposed by Gord): 1) illegal 2) 100% coverage 3) 0% coverage 4) eliminate from coverage The reason for the 4th option is that an empty coverpoint can be simultaneously considered as being trivially covered and not covered. In other words, we are devising semantics to resolve the mathematical indeterminacy of dividing zero by zero, which could indeed be 0 or 1. Since I don't know how we could apply L'Hopital's rule to coverage, I believe the best thing is to remove it from coverage. By that I mean the following: 1) An empty coverpoint (one that consists only of empty bins) does not affect the coverage score of the covergoup, that means, remove it from consideration from both the numerator and the denominator. 2) An empty covergroup (one that consists entirely of empty coverpoints) does not affect overall coverage score. A reporting tool should probably show these empty cover points/groups differently. I think that empty coverpoints are equivalent to unreachable states in state coverage (or unreachable lines in line coverage), and likewise, they should not modify the coverage score. Arturo -----Original Message----- From: owner-sv-ec@eda.org [mailto:owner-sv-ec@eda.org] On Behalf Of Gordon Vreugdenhil Sent: Wednesday, January 18, 2006 7:18 AM To: Swapnajit Chakraborti Cc: sv-ec@eda.org Subject: Re: [sv-ec] Completing my coverage thought... Swapnajit Chakraborti wrote: > > Gord, > > Thanks for clarifying the context. According to you, one scenario that > can result "empty" bins is the interaction of ignore_bins with other > bins. > Can you please elaborate a little more on this? > Do you mean that in the following example, bin b1 will be interpreted as > > an "empty" bin? Based on the recent discussion, the answer is yes. > coverpoint cp1 { > bins b1 = {0, 1, 2}; > bins b2[] = {0, 3, [7:15] }; > ignore_bins ig1 = {0, 1, 2}; > } > > If that is the case, then ignore_bins is actually affecting the total > coverage > space. Is that the desired behavior? In this context, I am attaching > below > an earlier discussion, where Arturo wrote to one of my queries that > ignore_bins will not > affect the total coverage space. Arturo, can you respond on this? Based on what we discussed, it seems that the coverage in the example that Swapnajit had asked about earlier should in fact be 100%. Gord. > > Arturo Salz wrote: > >>Swapnajit, >> >>An ignored coverbin does not modify the coverage space, it only > > indicates > >>that the specified bins not be counted. >>Thus, your example would do the following: >> - Create 7 auto-bins (0..6). >> - Bins 0, 1, and 2 never take a hit. >>And, if the sampled "a" takes on all values 0-7, the coverage is 4/7 > > => 57.1% > >> Answer > > > And here is the relevant example referred in Arturo's comment: > > >> reg [2:0] a; >> coverpoint a { >> ignore_bins ig_bins = {0, 1, 2}; >> } > > > Please let me know your comments on this. I do agree that if ignore_bins > does not > affect total coverage space, we can never reach 100% coverage. These > points > need to be clarified. > > The other point you mentioned that "empty" bins can result if there are > more > bins than values. I found this in LRM as well and clearly there is no > mention > whether to include them in coverage calculation. But according to me, > since > empty bins resulted from user specification, we should consider them for > coverage calculation. So, if user specifies following: > > reg [2:0] a; > coverpoint a { > bins b1[10] = {[0:7]}; > } > > And values reached during simulation are 0:7, the coverage will be > reported as 8/10, considering the "empty" bins, namely b1[8] and b1[9] > in the total. > > Thx, > Swapnajit. > > > > >>-----Original Message----- >>From: owner-sv-ec@eda.org [mailto:owner-sv-ec@eda.org] On Behalf Of >>Gordon Vreugdenhil >>Sent: Tuesday, January 17, 2006 11:18 PM >>To: Swapnajit Chakraborti >>Cc: Arturo Salz; SV_EC List >>Subject: Re: [sv-ec] Completing my coverage thought... >> >>Swapnajit, >> >>The discussion we were having was a continuation of the SV-EC >>conference call which is why you misunderstood this. >> >>By "empty bins" here we did not mean "bins that were never covered" but > > >>rather "bins with no elements in their value set". >>This arises due to the interaction of ignore_bins and/or having more >>bins than values. >> >>The LRM has not defined whether bins that have no elements in their >>value set are supposed to be included in the computation. >>Arturo and I are in agreement that they should not. As I noted in my >>follow-up email, there is still edge case behavior to discuss - the >>situation in which there are no bins with values in their value sets. >> >>Gord. >> >> >> >>Swapnajit Chakraborti wrote: >> >> >>>Hi Arturo & Gord, >>> >>>As per LRM, coverage for a coverpoint (section 18.10.1) is >> >>defined as: >> >>> num_covered_bins/num_all_defined_bins >>> >>>Again, num_covered_bins is actually the number of bins for which >>>at_least count is reached (count >= at_least). >>> >>>Now, in your discussion you have mentioned "num_non_empty_bins". >>>I am assuming the following definition of "num_non_empty_bins". >>> >>>num_non_empty_bins = num_covered_bins ( count >= at_least) + >>> (number of bins which have 0 < count < at_least) >>> >>>Is the above definition correct? >>> >>>In that case, if we define coverage as follows (as you mentioned): >>> >>> >>>>> num_covered_bins / num_non_empty_bins >>> >>> >>>We are completely removing the number of empty bins from the >>>denominator. >>> >>>Shouldn't the denominator be the total number of bins defined for a >>>coverpoint which is: >>>num_all_defined_bins = num_non_empty_bins + num_empty_bins >>> >>>Note that num_all_defined_bins will not include ignore/illegal bins. >>> >>>Thx, >>>Swapnajit. >>> >>>Gordon Vreugdenhil wrote: >>> >>> >>>>Arturo Salz wrote: >>>> >>>> >>>>>Gord, >>>>> >>>>>I agree that it's best to not consider empty bins and compute >>>>>coverage >>>>>as: >>>>> num_covered_bins / num_non_empty_bins While both mechanisms >>>>>described by you allow coverage to be 100%, this one also exhibits >>>>>the nice property of being 0 when nothing is run. I believe it is >>>>>counterintuitive to have non-zero coverage when no bins are covered. >>>>> >>>>> Arturo >>>> >>>>Arturo, >>>> >>>>That's fine with me. An edge case -- 18.10.1 doesn't define >> >>any rules >> >>>>for results when the number of non-empty bins is 0. So, should this >>>>be defined to be: >>>> 1) illegal >>>> 2) 100% coverage >>>> 3) 0% coverage >>>> >>>>I don't think I would want (1) to be an LRM requirement (an >>>>implementation might warn of course), but for parameterized >>>>ignore_bins, etc. the situation might actually be reasonable. >>>> >>>>If the user anticipates the scenario, I think that (2) is the best >>>>choice since otherwise full coverage can't be reached. >>>> >>>>Gord. >>>> >>>> >>>> >>>>>-----Original Message----- >>>>>From: owner-sv-ec@eda.org [mailto:owner-sv-ec@eda.org] On Behalf Of >>>>>Gordon Vreugdenhil >>>>>Sent: Monday, January 09, 2006 12:52 PM >>>>>To: SV_EC List >>>>>Subject: [sv-ec] Completing my coverage thought... >>>>> >>>>>Just to complete my thought regarding coverage before the call was >>>>>dropped -- my basic question is whether the coverage percentage is: >>>>> num_covered_bins / num_non_empty_bins >>>>>or: >>>>> (num_covered_bins + num_empty_bins) / (num_non_empty_bins + >>>>>num_empty_bins) >>>>> >>>>>(i.e. is an empty bin not considered at all or is it trivially >>>>>covered?) >>>>> >>>>>I think that the former is perferable, but I don't know for sure. >>>>> >>>>>Gord. >>>> >>>>-- >>>>-------------------------------------------------------------------- >>>>Gordon Vreugdenhil 503-685-0808 >>>>Model Technology (Mentor Graphics) gordonv@model.com >> >>-- >>-------------------------------------------------------------------- >>Gordon Vreugdenhil 503-685-0808 >>Model Technology (Mentor Graphics) gordonv@model.com >> > > > -- -------------------------------------------------------------------- Gordon Vreugdenhil 503-685-0808 Model Technology (Mentor Graphics) gordonv@model.comReceived on Fri Jan 20 11:05:28 2006
This archive was generated by hypermail 2.1.8 : Fri Jan 20 2006 - 11:06:06 PST