More than one name in Conventions attribute

[Dave.Allured]

Introduction

It is proposed to support the listing of more than one convention name in the Conventions global attribute. As of ​CF-1.6, only the CF identification by itself is allowed. The option of a list has been in Unidata's recommendations for a long time; see references. There was some community support for this in the ​recent mailing list discussion.

Process

I would like to present this in two parts. This trac ticket is part 1, which proposes to adopt verbatim, the Unidata language for listing more than one convention name. The original Unidata recommendation allows the list of names to be separated by either spaces or commas.

Part 2, which I plan to file as a separate ticket, would propose more detailed parsing rules and would probably replace the syntax description of part 1. The main objectives of part 2 would be to accommodate both unambiguous machine parsing, and flexibility in allowable characters in convention names.

Why two tickets? I think that part 1 has a better chance of approval. It accomplishes the most important objective of multiple conventions in a workable way. If part 2 succeeds, it can replace part 1, sooner or later.

The ​mailing list discussion beginning December 2011 should be considered part of the record for this proposal.

Text of proposed changes

Two new paragraphs are to be added at the end of the current section 2.6.1, Identification of Conventions:

It is possible for a netCDF file to adhere to more than one set of conventions, even when there is no inheritance relationship among the conventions. In this case, the value of the Conventions attribute may be a single text string containing a list of the convention names separated by blank space (recommended) or commas (if a convention name contains blanks). This is the Unidata recommended syntax from ​NetCDF Users Guide, Appendix B.

When CF is listed with other conventions, this asserts the same full compliance with CF requirements and interpretations as if CF was the sole convention. It is the responsibility of the data-writer to ensure that all common metadata is used with consistent meaning between conventions.

In the conformance document, add a new second line under 2.6.1 Identification of Conventions:

• The Conventions attribute may be a single text string containing a list of convention names separated by blank space or commas, one of which shall be the full CF string as described above.

References

​Current language of section 2.6.1, Identification of Conventions (CF-1.6)

​NetCDF Users Guide, Appendix B, Attribute Conventions

​List of registered conventions, NetCDF website

​Ticket #27, Use namespace tags to include other conventions, 2008
(Previous proposal and discussion of Conventions attribute, not yet adopted)

[jonathan]

Dear Martin

Thanks for proposing this, which looks reasonable to me. Perhaps we ought to be clearer about the alternative parsing rules, to avoid ambiguity, for instance like this: If the string contains any commas, it is assumed to be a comma-separated list; if it contains no commas, it is interpreted as a blank-separated list.

I am glad to see the requirement of CF compliance.

Cheers

Jonathan

[jonathan]

Sorry, Dave - not Martin! I confused the tickets.

[pbentley]

Hi Dave,

Since all netCDF attributes are treated as vectors, would an alternative proposition - one that might side-step the comma/space schism - be to stipulate that multiple conventions should be defined as separate string elements, thus:

   :Conventions = "CF-1.5", "ACDD v1.0", "Dublin Core", ... ;

(The NCO and CDO tools do something similar with the history attribute.)

If not, then standardising on the comma character as the *only* separator would get my vote, on the basis that few (if any) standard acronyms will contain that character.

Phil

[russ]

Hi Phil,

Unfortunately, netCDF-3 attribute vectors are merely vectors of characters, not strings.

So supporting a list of multiple conventions using a vector of separate string elements would work fine for netCDF-4, which has "string" as a primitive type, but netCDF-3 has no established convention for delimiting strings in a vector of characters.

--Russ

[russ]

Martin,

I support this proposal. With the other support expressed and the lack of objections, this makes it eligible for acceptance.

--Russ

[jonathan]

I agree. This change should be implemented in the next version of CF, and Dave's name should then be added to the list of contributing authors.

No-one objected to my first comment, so I presume it too is accepted, namely to append this sentence to the end of Dave's first proposed paragraph. "If the string contains any commas, it is assumed to be a comma-separated list."

Jonathan

[ngalbraith]

I'd like to see the text (2.6.1. Identification of Conventions) changed to something simple, and let users refer to the NetCDF Conventions page for more details. Something like:

We recommend that netCDF files that follow these conventions indicate this by setting the NUG defined global attribute Conventions to the string value "CF-1.6". For files that adhere to more than one set of conventions, the value of the Conventions attribute may be a string containing a list of the convention names separated by blank space (recommended) or commas (if a convention name contains blanks), for example

:Conventions = "CF-1.6 ACDD-1.0" ;

More information is available on the NetCDF Conventions page at ​http://www.unidata.ucar.edu/software/netcdf/conventions.html'

I'm hoping that this will be incorporated soon, because presently the CF checkers don't accept multiple terms in the Conventions attribute. And so, for example, the NetCDF Attribute Convention for Dataset Discovery (ACDD) uses a different attribute (metadata_conventions) to identify ACDD-compliant files.

This could get very messy - so much better to allow all the conventions to be listed in one attribute.

[jonathan]

Dear Nan

This proposal has already been accepted, so it'll be included in the agreed form in the next version of CF. Is that OK? The agreed text is this (from the above) to add at the end of Sect 2.6.1:

It is possible for a netCDF file to adhere to more than one set of conventions, even when there is no inheritance relationship among the conventions. In this case, the value of the Conventions attribute may be a single text string containing a list of the convention names separated by blank space (recommended) or commas (if a convention name contains blanks). This is the Unidata recommended syntax from NetCDF Users Guide, Appendix B. If the string contains any commas, it is assumed to be a comma-separated list.

When CF is listed with other conventions, this asserts the same full compliance with CF requirements and interpretations as if CF was the sole convention. It is the responsibility of the data-writer to ensure that all common metadata is used with consistent meaning between conventions.

There is a corresponding change in the conformance document, given at the top of this ticket.

I agree that an example would be helpful, and your example could be inserted.

Jonathan

[ngalbraith]

That's great, thanks. In part, I brought this up because I was hoping that the text pertaining to the ftp directory structure ('The string is interpreted as a directory name ...') would be removed, so I'm glad to see that it's gone.

Now guess it's just a matter of waiting for the next release, and maybe encouraging people to pre-adopt this syntax, although it produces errors on the CF checkers.

[painter1]

This ticket, #76, has now been implemented in the CF Conventions document as described in comment 8. However the ticket cannot yet be closed because the conformance document has not been updated.