Re: [sv-ec] Mantis proposal for coverage computation

From: David Scott <david_scott_at_.....>
Date: Fri Jul 20 2007 - 16:17:01 PDT
Arturo -- and others --

Our proposal easily and simply accommodates crosses, illegal bins, and so forth; I'll describe the intuitive basis in a moment with some examples.

By contrast, the sub-type looks like an entirely new sort of coverage concept without the first-class status of either the covergroup type or covergroup instance.  For example, covergroup types have their type_option structures, as covergroup instances have their option structures; there are specific methods that apply to each.  Introducing a sub-type without full support for options and query mechanisms is not, I think, a good idea -- and that is a major language change.

The sub-type is not implied by the LRM.  2008 Draft 3a contains this, unchanged from 2005:

It is important to understand that the cumulative coverage considers the union of all significant bins; thus, it
includes the contribution of all bins (including overlapping bins) of all instances.

This implies that the cumulative (type) coverage is a union of coverage from all bins of all instances.  The only notion that remains vaguely specified is what is an "overlapping bin" among instances.  A union operation requires an identity function, so that you can determine which members belong to multiple sets and thus are added only once to the union.

Intuitively, our proposal -- initially devised by Ray Ryan, written up and fully specified by myself -- is to consider coverage bins the same that are named the same.   This relies on names implied in the current LRM except for one case which I'll get to at the end of this e-mail.

Whether it is appropriate to merge different coverage spaces (you say "coverage grids") is a matter that users may decide in some cases.  There is some -- though not perhaps complete -- facility for users to specify this by the way bins are declared.

So let's take this as my first example:

    // Example 1: fixed bin names
    covergroup cg (int l,r);
        option.per_instance = 1;
        coverpoint p1 {
            bins p1 = { [l:r] };
        }
        coverpoint p2 {
            bins p2 = { [l:r] };
        }
        x: cross p1, p2;
    endgroup
    cg cv1 = new(0,1);
    cg cv2 = new(1,2);

Here, while the coverage space of each instance is different, they are declared in a fixed way, with bin names "p1" and "p2".  If we use the bin name as the criterion for merging, it has the virtue of simplicity: things that are named the same get merged together from the instances.  The LRM even specifies the name of the cross bin (by example in 18.6 and 18.6.1): "<p1,p2>".  These names can be used unambiguously to compute the merge.  Consider this sampling scenario where I've avoided covering the cross to illustrate the case of each instance covering something different:

        p1 = 0; p2 = -1;
        cv1.sample;
        p2 = 2; p1 = -1;
        cv2.sample;
        $display(cv1.get_inst_coverage());
        $display(cv2.get_inst_coverage());
        $display(cg::get_coverage());

Since p1 is covered by cv1 and p2 is covered by cv2, I expect output of 33.33...% for each instance but 66.66...% for the type.

In the following case, the bins are named according to the coverage space:

    // Example 2: bins named according to coverage space
    covergroup cg (int l,r);
        option.per_instance = 1;
        coverpoint p1 {
            bins p1[] = { [l:r] };
        }
        coverpoint p2 {
            bins p2[] = { [l:r] };
        }
        x: cross p1, p2;
    endgroup

    cg cv1 = new(0,1);
    cg cv2 = new(1,2);

Again, by example in 18.5 and 18.6, I have enough information to conclude what are the bin names.  I'll enumerate them:

In this case, it obvious what bins belong to what scope, but needless to say bins are only merged when they appear in the same scope.  In this case, the union is:

So the overlapping bins are p1[1], p2[1], and the corresponding cross-product bin <p1[1],p2[1]>.  If any of these three bins are covered in either instance, the cumulative (type) is computed with that bin covered -- which is only to say that the coverage is merged from the two instances.  Explicitly declared cross bins are an even easier case than this, because those always have a single name (the array [] syntax is not allowed.)

This even extends simply to illegal and ignore bins, for example:

    // Example 3: with ignore bins
    covergroup cg (int l,r,i);
        option.per_instance = 1;
        coverpoint p  {
            bins p[] = { [l:r] };
            ignore_bins ig = { i };
        }
    endgroup
    cg cv1 = new(0,2,1);
    cg cv2 = new(0,2,2);

In this case, the instance referenced by cv1 has coverage bins p[0] and p[2], while the instance referenced by cv2 has bins p[0] and p[1].  The bins p[0] overlap and get merged together when computing cumulative (type) coverage, and the type effectively has 3 bins.

The problematic case is the bin declared as an "array" with a "size" (here the "size" is 2):
bins p[2] = { l, r };
bins q[2] = { [l:r] };
If l==r, the first case creates 2 bins with the same value.  (The second case does not, because the range [l:r] is equivalent to r for l==r, and that is just a single value; the values must be separate members of the list to imply creation of two bins with the same associated value.)  Since the user specified this for some reason, we must make a distinction between the two bins.  There is no language in the current LRM that refers to bins in this case.  Ray's suggestion is always to name the bins by their array index, i.e., p[0] ... p[n-1] where n is the expression in square brackets in the declaration.  The user has implied the grouping by name, in that there are always n bins with some possibly parameterized set of values for each.  This is more similar to the case of a single bin name (declared as "bins p") with potentially multiple values than the bins which are associated with unique values (declared as "bins p[]").


So I hope you all forgive me the length -- but that describes our proposal with examples.  It is easy to describe, introduces no fundamentally new concepts to the LRM, and follows as a clarification of what is already there.  Perhaps Ray can address the motivation more eloquently than I can.

I'm hoping this is on Monday's agenda?

Cheers,

Dave Scott, Mentor Graphics


Arturo Salz wrote:

Dave,

 

I find this proposal problematic.

 

First, the proposal  is incomplete since it only deals with cover-points and doesn't specify how to handle cross-points of the merged cover-points. Consider for example:

 

CG_1 {              // One instance of covergroup CG

Cp1: b1_1, b1_2;           // cover point with two bins

Cp2: b2_1; b2_2;           // cover point with two bins

Cr: Cp1xCp2;                // Cross with 4 bins

}

 

CG_2 {              // Another instance of covergroup CG

Cp1: b1_3, b1_4;           // cover point with two bins

Cp2: b2_3; b2_4;           // cover point with two bins

Cr: Cp1xCp2;                // Cross with 4 bins

}

 

How many cross-bins are to be considered for the cumulative coverage instance? 8 (add => 4+4) or 16 (multiply => 4x4)? Changing the bin space of a cover point impacts all such transitive dependencies. And, both add and multiply schemes suffer from similar problems.

 

At a more fundamental level, this proposal requires tools to lose information that users felt was important to specify, collect, and hence report. In general, merging coverage data of a particular cover-group across multiple instances or even tests requires a consistent coverage grid. Parameterization of cover-groups can lead to the following inconsistencies in the coverage grid:

1.       Differences in the number of bins (for a given coverage item), due to:

a.       Differences in auto_bin_max

b.      Bin elimination due to illegal/ignore bins

c.       Unconstrained/constraint array bins generation  

d.      Different cross spaces

2.       Differences in bin names (for a given coverage item)

a.       Generated array bins names differ

3.       Differences in bin coverage space

a.       Arguments are used to modify the extent of the coverage space

 

A better scheme to address this problem is to dynamically create separate sub-types for differently parameterized cover-groups. The cumulative coverage of the cover-group is then be computed as the average coverage of all its sub-types. It is better to dynamically create new cover-group subtypes than dynamically modify an existing type. This way users have access to all the specific information as well as the cumulative (or merged) information.

 

Note that in general, parameterization that changes the bin-space though arguments is not a good methodology because it complicates the merging and the user interpretation of coverage data resulting from multiple tests.

 

I’ve added this email as a note to your proposal.

 

            Arturo             

 

-----Original Message-----
From: owner-sv-ec@eda.org [mailto:owner-sv-ec@eda.org] On Behalf Of David Scott
Sent: Friday, June 22, 2007 11:00 AM
To: sv-ec@eda-stds.org
Subject: [sv-ec] Mantis proposal for coverage computation

 

I've submitted a new Mantis against coverpoint coverage computation:

 

http://eda.org/svdb/view.php?id=1897

 

This is to clarify the language "union of all significant bins" and

"overlapping bins" -- which is relevant to computing cumulative coverage

for covergroups with parameterized instances.

 

-- Dave Scott

 

 

--

This message has been scanned for viruses and

dangerous content by MailScanner, and is

believed to be clean.

 



--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean. Received on Fri Jul 20 16:17:29 2007

This archive was generated by hypermail 2.1.8 : Fri Jul 20 2007 - 16:18:05 PDT