Re: [sv-ec] Completing my coverage thought...

Swapnajit Chakraborti wrote:

>  
> Gord,
> 
> Thanks for clarifying the context. According to you, one scenario that
> can result "empty" bins is the interaction of ignore_bins with other
> bins.
> Can you please elaborate a little more on this? 
> Do you mean that in the following example, bin b1 will be interpreted as
> 
> an "empty" bin? 

Based on the recent discussion, the answer is yes.


> coverpoint cp1 {
>    bins b1 = {0, 1, 2};
>    bins b2[] = {0, 3, [7:15] };
>    ignore_bins ig1 = {0, 1, 2};
> } 
> 
> If that is the case, then ignore_bins is actually affecting the total
> coverage
> space. Is that the desired behavior? In this context, I am attaching
> below
> an earlier discussion, where Arturo wrote to one of my queries that
> ignore_bins will not
> affect the total coverage space.

Arturo, can you respond on this?  Based on what we discussed,
it seems that the coverage in the example that Swapnajit had
asked about earlier should in fact be 100%.

Gord.


> 
> Arturo Salz wrote:
> 
>>Swapnajit,
>>
>>An ignored coverbin does not modify the coverage space, it only
> 
> indicates
> 
>>that the specified bins not be counted.
>>Thus, your example would do the following:
>>    - Create 7 auto-bins (0..6).
>>    - Bins 0, 1, and 2 never take a hit.
>>And, if the sampled "a" takes on all values 0-7, the coverage is 4/7
> 
> => 57.1%
> 
>>    Answer
> 
> 
> And here is the relevant example referred in Arturo's comment:
> 
> 
>>  reg [2:0] a;
>>  coverpoint a {
>>     ignore_bins ig_bins = {0, 1, 2};
>>  }
> 
> 
> Please let me know your comments on this. I do agree that if ignore_bins
> does not
> affect total coverage space, we can never reach 100% coverage. These
> points
> need to be clarified.
> 
> The other point you mentioned that "empty" bins can result if there are
> more 
> bins than values. I found this in LRM as well and clearly there is no
> mention
> whether to include them in coverage calculation. But according to me,
> since 
> empty bins resulted from user specification, we should consider them for
> coverage calculation. So, if user specifies following:
> 
> reg [2:0] a;
> coverpoint a {
>   bins b1[10] = {[0:7]};
> }
> 
> And values reached during simulation are 0:7, the coverage will be
> reported as 8/10, considering the "empty" bins, namely b1[8] and b1[9]
> in the total.
>  
> Thx,
> Swapnajit.
> 
> 
> 
> 
>>-----Original Message-----
>>From: owner-sv-ec@eda.org [mailto:owner-sv-ec@eda.org] On Behalf Of 
>>Gordon Vreugdenhil
>>Sent: Tuesday, January 17, 2006 11:18 PM
>>To: Swapnajit Chakraborti
>>Cc: Arturo Salz; SV_EC List
>>Subject: Re: [sv-ec] Completing my coverage thought...
>>
>>Swapnajit,
>>
>>The discussion we were having was a continuation of the SV-EC 
>>conference call which is why you misunderstood this.
>>
>>By "empty bins" here we did not mean "bins that were never covered" but
> 
> 
>>rather "bins with no elements in their value set".
>>This arises due to the interaction of ignore_bins and/or having more 
>>bins than values.
>>
>>The LRM has not defined whether bins that have no elements in their 
>>value set are supposed to be included in the computation.
>>Arturo and I are in agreement that they should not.  As I noted in my 
>>follow-up email, there is still edge case behavior to discuss - the 
>>situation in which there are no bins with values in their value sets.
>>
>>Gord.
>>
>>
>>
>>Swapnajit Chakraborti wrote:
>>
>>
>>>Hi Arturo & Gord,
>>>
>>>As per LRM, coverage for a coverpoint (section 18.10.1) is
>>
>>defined as:
>>
>>>  num_covered_bins/num_all_defined_bins
>>>
>>>Again, num_covered_bins is actually the number of bins for which 
>>>at_least count is reached (count >= at_least).
>>>
>>>Now, in your discussion you have mentioned "num_non_empty_bins".
>>>I am assuming the following definition of "num_non_empty_bins".
>>>
>>>num_non_empty_bins = num_covered_bins ( count >= at_least) + 
>>>                     (number of bins which have 0 < count < at_least)
>>>
>>>Is the above definition correct?
>>>
>>>In that case, if we define coverage as follows (as you mentioned):
>>>
>>>
>>>>>     num_covered_bins / num_non_empty_bins
>>>
>>>
>>>We are completely removing the number of empty bins from the 
>>>denominator.
>>>
>>>Shouldn't the denominator be the total number of bins defined for a 
>>>coverpoint which is:
>>>num_all_defined_bins = num_non_empty_bins + num_empty_bins
>>>
>>>Note that num_all_defined_bins will not include ignore/illegal bins.
>>>
>>>Thx,
>>>Swapnajit.
>>>
>>>Gordon Vreugdenhil wrote:
>>>
>>>
>>>>Arturo Salz wrote:
>>>>
>>>>
>>>>>Gord,
>>>>>
>>>>>I agree that it's best to not consider empty bins and compute 
>>>>>coverage
>>>>>as:
>>>>>     num_covered_bins / num_non_empty_bins While both mechanisms 
>>>>>described by you allow coverage to be 100%, this one also exhibits 
>>>>>the nice property of being 0 when nothing is run. I believe it is 
>>>>>counterintuitive to have non-zero coverage when no bins are covered.
>>>>>
>>>>>     Arturo
>>>>
>>>>Arturo,
>>>>
>>>>That's fine with me.  An edge case -- 18.10.1 doesn't define
>>
>>any rules
>>
>>>>for results when the number of non-empty bins is 0.  So, should this 
>>>>be defined to be:
>>>>  1) illegal
>>>>  2) 100% coverage
>>>>  3) 0% coverage
>>>>
>>>>I don't think I would want (1) to be an LRM requirement (an 
>>>>implementation might warn of course), but for parameterized 
>>>>ignore_bins, etc. the situation might actually be reasonable.
>>>>
>>>>If the user anticipates the scenario, I think that (2) is the best 
>>>>choice since otherwise full coverage can't be reached.
>>>>
>>>>Gord.
>>>>
>>>>
>>>>
>>>>>-----Original Message-----
>>>>>From: owner-sv-ec@eda.org [mailto:owner-sv-ec@eda.org] On Behalf Of 
>>>>>Gordon Vreugdenhil
>>>>>Sent: Monday, January 09, 2006 12:52 PM
>>>>>To: SV_EC List
>>>>>Subject: [sv-ec] Completing my coverage thought...
>>>>>
>>>>>Just to complete my thought regarding coverage before the call was 
>>>>>dropped -- my basic question is whether the coverage percentage is:
>>>>>    num_covered_bins / num_non_empty_bins
>>>>>or:
>>>>>    (num_covered_bins + num_empty_bins) / (num_non_empty_bins +
>>>>>num_empty_bins)
>>>>>
>>>>>(i.e. is an empty bin not considered at all or is it trivially
>>>>>covered?)
>>>>>
>>>>>I think that the former is perferable, but I don't know for sure.
>>>>>
>>>>>Gord.
>>>>
>>>>--
>>>>--------------------------------------------------------------------
>>>>Gordon Vreugdenhil                                503-685-0808
>>>>Model Technology (Mentor Graphics)                gordonv@model.com
>>
>>--
>>--------------------------------------------------------------------
>>Gordon Vreugdenhil                                503-685-0808
>>Model Technology (Mentor Graphics)                gordonv@model.com
>>
> 
> 
> 

-- 
--------------------------------------------------------------------
Gordon Vreugdenhil                                503-685-0808
Model Technology (Mentor Graphics)                gordonv@model.com