NetCDF SWG

 

TITLE: NetCDF Standards Working Group Charter
Author Name (s): Ben Domenico
Email: ben [at] ucar.edu
DATE: 28 April 2014
CATEGORY: Data Model and Encoding Standard Working Group
 

To:  OGC members & interested parties A new OGC Standards Working Group is being formed. The OGC members listed below have proposed the OGC NetCDF Standards Working Group (SWG).  The SWG proposal provided in this document meets the requirements of the OGC TC Policies and Procedures.
The SWG name, statement of purpose, scope, list of deliverables, audience, and language specified in the proposal will constitute the SWG's official charter. Technical discussions may occur no sooner than the SWG's first meeting. This SWG will operate under the OGC 2007 IPR Policy. The eligibility requirements for becoming a participant in the SWG at the first meeting (see details below) are that:

  • You must be an employee of an OGC member organization or an individual member of OGC;
  • The OGC member must have signed the OGC Membership agreement;
  • You must notify the SWG chair of your intent to participate to the first meeting. Members may do so by logging onto the OGC Portal and navigating to the Observer page and clicking on the link for the SWG they wish to join and;
  • You must attend meetings of the SWG. The first meeting of this SWG is at the time and date fixed below. Attendance may be by teleconference.

Of course, participants also may join the SWG at any time. The OGC and the SWG welcomes all interested parties. Non-OGC members who wish to participate may contact us about joining the OGC. In addition, the public may access some of the resources maintained for each SWG: the SWG public description, the SWG Charter, Change Requests, and public comments, which will be linked from the SWG’s page. Please feel free to forward this announcement to any other appropriate lists. The OGC is an open standards organization; we encourage your feedback.
 
 
 
 
 

1.    NetCDF Standards Working Group

 

2.    Purpose of this Standards Working Group

NetCDF has already been established as an adopted OGC standard, encompassing a core standard along with extensions for specific data models and encodings and for the Climate and Forecast (CF) metadata conventions.
The purpose of this Standards Working Group is to extend further the existing netCDF standard with extension modules for additional data models, encodings, and conventions.  The additional extensions include, but are not limited to, those currently under consideration by the currently existing CF-netCDF 1.0 SWG, which will be disbanded and replaced by this NetCDF SWG.

3.    Business Value Proposition

NetCDF (network Common Data Form) is a data model for multidimensional array-oriented scientific data, a freely distributed collection of access libraries implementing support for that data model, and a machine-independent storage format. Together, the interfaces, libraries, and format support the creation, access, and sharing of scientific data.
NetCDF has been formally recognized by US Government standards bodies including:
·         the NASA Earth Standards Data Systems Working Groups (ESDSWG) Standards Process Group (SWG),
·         the Data Management and Communications Committee of the NOAA Integrated Ocean Observing System, and
·         the Digital Preservation Program of the US Library of Congress.
The goal of this standards effort is to have netCDF along with its associated conventions recognized as an international standard for encoding georeferenced data in binary form.
The OGC has developed a broad Standards Baseline. UCAR and the OGC believe that having netCDF included in that family will encourage broader use and greater interoperability among clients and servers interchanging data in binary form. Having already established netCDF as a standard for binary encoding has made it possible to incorporate standard delivery of data in binary form via several OGC protocols, e.g., WCS, WFS, and SOS.  Establishing additional extensions specifications to the core  netCDF encoding standard will accelerate that process and allow for more encoding flexibility. Work is already underway on extension to GML and OWS for delivery of data encoded in netCDF.
Additional netCDF data model extensions will be considered to improve the effectiveness and usability of netCDF datasets for OGC Community. One example is  a recently released discussion paper on uncertainty conventions.

4.    Scope of Work

This SWG is focused on processing netCDF candidate standard submissions for additional data models, encodings, and conventions, coordinating the public comment periods for the proposed extensions, and processing any comments received during the public comment periods.   The final deliverables of the SWG will be additional extensions to the existing netCDF core to be submitted for consideration by the membership.
 
4.1    Statement of relationship of planned work to the current OGC standards baseline
The currently existing netCDF standards and a related primer document are given in the table below:

Version

Document Title (click to download)

Document #

Type

 1.0

OGC Network Common Data Form (NetCDF) Core Encoding Standard version 1.0

 10-090r3 

IS

 

NetCDF Binary Encoding Extension Standard: NetCDF Classic and 64-bit Offset Format (1.0)

 10-092r3 

ISx

 

CF-netCDF3 Data Model Extension standard (3.1)

 11-165r2 

IS

 

OGC Network Common Data Form (NetCDF) NetCDF Enhanced Data Model Extension Standard (1.0)

 11-038R2 

IS

 

NetCDF Uncertainty Conventions

 11-163 

DP

 

CF-netCDF Core and Extensions Primer

 10-091r3 

Primer

 
At present, the CF-netCDF 1.0 SWG is discussing further extensions to the netCDF core for:

  • netCDF-Uncertainty conventions;
  • CF-netCDF 3.0 encoding using GML 3.2.1 Application Schema -Coverages
  • NetCDF Markup Language (NcML) encoding specification;
  • NcML-GML encoding specification;
  • Extension to OWS Common for CF-netCDF data exchange;
  • Extension to WCS 2.0 core DescribeCoverage  for CF-netCDF data description.

 
The NetCDF SWG takes over the future development of these extension standards and needed updates to the overview primer document.  However, the scope of the SWG is not limited to the extensions currently under consideration.  The scope includes any proper extensions to the existing core netCDF standard and extensions to other OGC standards that specify netCDF encoding.  The SWG membership will determine which extensions are proper.
 
4.2    What is Out of Scope?
Only extensions to the existing core netCDF standard and extensions to other OGC standards that specify netCDF encoding will be considered by the netCDF SWG.  Anything else is considered out of scope.
4.3    Specific Contribution of Existing Work as a Starting Point
Related netCDF standards recognized by other organizations are listed in section 3.0.  Existing OGC netCDF-related standards are listed in section 4.1.  The work of the CF-netCDF 1.0 SWG is the starting point for this SWG.
4.4    Determination of SWG Completion
 
The SWG will dissolve when there are:
·         no additional extensions to the netCDF standard under consideration, and
·         no extensions to OGC data delivery standard for netCDF encoding under consideration. 
 
This SWG is a persistent SWG.

5.    Description of Deliverables

An updated version of the CF-netCDF Core and Extensions Primer will be maintained as a description of current and anticipated deliverables along with a chart indicating which extensions are under consideration at any given time.  The Primer will be renamed “NetCDF Core and Extensions Primer.”

CF-netCDF Core and Extensions Primer

 10-091r3 

Primer

 

6.    IPR Policy for this SWG

X RAND-Royalty Free.    RAND for fee

7.    Anticipated Participants

This SWG will start with the current membership of the CF-netCDF 1.0 SWG.

8.    Other Informative Remarks about this SWG

a. Similar or Applicable Standards Work (OGC and Elsewhere). For example:
Related netCDF standards recognized by other organizations are listed in section 3.0.  Existing OGC netCDF-related standards are listed in section 4.1.  The work of the CF-netCDF 1.0 SWG is the starting point for this SWG.
 
The NetCDF SWG also maintains close liaison relationships with the OPeNDAP and HDF communities as well as the CF Conventions community and the OGC working groups developing standards for data delivery, e.g., WCS, WFS, SOS, OWS Common.
b. Details of the First Meeting The first meeting will be at the OGC TC meeting at the WMO in Geneva, June 2014. c. Projected On-going Meeting Schedule
 The SWG will hold sessions at each OGC TC meeting when there are sufficient attendees.  Email interactions and teleconference meetings will be held as needed.
d. Supporters of the Proposal The following people support this proposal and are committed to the Charter and projected meeting schedule. These members are known as SWG Founding or Charter members. Once the SWG is officially activated, this group is immediately “opted-into” the SWG and have voting rights from the first day the SWG is officially formed. Extend the table as necessary.

Name

Organization

The members in good standing of the CF-netCDF 1.0 SWG

Various

 

 

 

 

 
 
  e. Convener(s) Ben Domenico, Unidata (US) Stefano Nativi, CNR-IIA (Italy)
 
------------------------------------------------------------------------------------
Below is the description based on the original charter of the CF-netCDF 1.0 Standards Working Group. 
It has been superceded by the charter in the section above.
--------------------------------------------------------------------------------------
1.    NetCDF Standards Working Group
NetCDF (network Common Data Form) is a data model for array-oriented scientific data, a freely distributed collection of access libraries implementing support for that data model, and a machine-independent format. Together, the interfaces, libraries, and format support the creation, access, and sharing of scientific data.
NetCDF has been formally recognized by US Government standards bodies: the NASA Earth Standards Data Systems Working Groups (ESDSWG) Standards Process Group (SWG) and the Data Management and Communications Committee of the NOAA Integrated Ocean Observing System.  The goal of this standards effort is to have the NetCDF with CF (Climate and Forecast) Conventions recognized as an international standard for encoding georeferenced data in binary form.

Normal
0

false
false
false

EN-US
X-NONE
X-NONE

/* Style Definitions */
table.MsoNormalTable
{mso-style-name:"Table Normal";
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-priority:99;
mso-style-parent:"";
mso-padding-alt:0in 5.4pt 0in 5.4pt;
mso-para-margin:0in;
mso-para-margin-bottom:.0001pt;
mso-pagination:widow-orphan;
font-size:10.0pt;
font-family:"Times New Roman","serif";}

The OGC has developed a broad Standards Baseline. UCAR and the OGC believe that having netCDF included in that family will encourage broader use and greater interoperability among clients and servers interchanging data in binary form. Having already established netCDF as a standard for binary encoding is making it possible to incorporate standard delivery of data in binary form via several OGC protocols, e.g., WCS, WFS, and SOS.  Establishing additional extensions specifications to the core  netCDF encoding standard will accelerate that process and allow for more encoding flexibility. Work is already underway on extension to GML and OWS for delivery of data encoded in netCDF.
Additional netCDF data model extensions will be considered to improve the effectiveness and usability of netCDF datasets for OGC Community. One example is  a recently released discussion paper on uncertainty conventions.
2.    Purpose of this Standards Working Group
The purpose of this Standards Working Group is to progress the netCDF with CF-conventions to the state of an adopted standard. This will be achieved in two or more stages. The first stage will establish the core standard.  The core standard will be based on the 09-122 discussion paper, "CF-netCDF Encoding Specification," submitting it as a candidate standard, processing the comments received during the public comment period, and ensuring that the standard is consistent with the OGC baseline and business plan and is backwards compatible with existing netCDF and CF documents and implementations already in widespread use within the mass market.   Another document covering the CF conventions will be submitted as a candidate extension standard and will follow the same process.   The core netCDF standard,  together with the CF extension standard,  are necessary elements of the CF-netCDF Encoding Specification.
Subsequently, additional extension standards covering additional extension standard such as ncML-GML metadata encoding, and  CF-netCDF Application Programming Interface (API) may be considered at the discretion  of the active SWG members.
3.    Scope of Work
This SWG is focused on processing the CF-netCDF candidate standard submissions for the core standard and one extension for the CF conventions, coordinating the public comment periods for the core and the CF extension, and processing any comments received during the public comment periods.   The final deliverable of the SWG will be a version of the candidate core standard and the CF extension for consideration by the membership.
3.1    What is out of scope?
This SWG will focus only on processing comments received during the 30 day public comment period for the core and CF convention extension. Subsequent Change Requests are therefore out of scope.
3.2    Specific Contribution of Existing Work as a Starting Point
The starting point for the work will be the draft CF-netCDF submission, which is based on the approved standard specification of the NASA Standards Process Group (SPG). This will be sent out for public comment following any edits required to meet candidate standard requirements.
3.3    How it is to be Determined when the Work of the SWG has been Completed?
The work of the SWG will be complete when:

  • All comments submitted during the 30 day public comment periods for the core standard and CF extension have been dealt with to the satisfaction of the SWG.
  • The SWG approves the CF-netCDF Candidate Core Standard and the CF extension standard , for submission to the TC for approval as an adopted standard.
  • The candidate standard has been approved by the OGC Technical and Planning Committees as an Adopted OGC standard.

4.    Description of deliverables
The following deliverables will result from the work of this SWG:

  • An annotated list of all comments submitted during the 30 day public comment periods for the core standard and for the CF extension -- including the comment, submitter, rationale, comment type/priority, and the response of the SWG.
  • A final version of the CF-netCDF Core Standard and the CF extension standard for submission to the TC.

Core Standard Milestones: The timeline for the core standard. In recognition of the rapid product development cycles inherent in the mass market community, the SWG should conclude its activities on the core standard no later than Q1 2010. As such the following schedule of activities is planned for the core standard:

  • SWG startup. 2009 October.
  • OAB RFC review; SWG prepares candidate standard for vote for public release. 2009 October
  • SWG votes to release CF-netCDF for public comment. 2009 November
  • OGC staff develops PR and announces availability for public comment. 2009 November
  • 30-day public comment period.2009 November - 2009 December
  • SWG collects and collates comments into a single document. 2009 December
  • SWG discusses and votes on comments and makes resulting edits towards final CF-netCDF candidate standard. January
  • SWG votes to approve release of CF-netCDF candidate standard for approval as an OGC adopted standard. 2009 February
  • 60 day IPR review period and associated TC e-vote to approve CF-netCDF candidate standard as .an OGC adopted standard. 2010 February - 2010 March

CF Extension Standard Milestones: A similar timeline is planned for the CF extension standard but it will be offset by 3 months.
Optional Additional Extensions: Additional extensions will be considered at the discretion of the SWG members.  Timelines will be submitted at that time.
 
5.    IPR Policy for this SWG
X RAND-Royalty Free. 
6.    Anticipated Participants
The target audience/participants of CF-netCDF includes search engine developers, GIS vendors, the geosciences research and education community, international government agencies that distribute and use global Earth observations and forecast, mass market software vendors interested in geographic based applications, and the consumer.
7.    Other informative information about the work of this SWG
a. Similar or applicable standards work (OGC and elsewhere). The following standards and projects may be relevant to the SWG's planned work, although none currently provide the functionality anticipated by this committee's deliverables:
NASA Earth Systems Data Systems (ESDS) Standards Process Group (SPG) RFC-011: NetCDF Classic and 64-bit Offset File Formats
ISO 19123
ISO 19111
OGC Geography Markup Language v3.2 (also ISO 19136)
OGC WCS
NASA SPG
The SWG intends to seek and if possible maintain liaison with each of the organizations maintaining the above works.
b. Details of the first meeting The first meeting of the committee will be held by telephone conference call at 8AM 2009 October 14. Call-in information will be provided to the SWG's e-mail list and on the portal calendar in advance of the meeting. c. Projected on-going meeting schedule
The work of the committee will be carried out primarily by email and conference calls, every other week, with face-to-face meetings at each of the OGC TC meetings. Other face to face meetings may be called if this can expedite the work of the SWG..   d. Supporters of the Proposal The following people support this proposal and are committed to the Charter and projected meeting schedule. These members are known as SWG Founding or Charter members. Once the SWG is officially activated, this group is immediately "opted-into" the SWG and have voting rights from the first day the SWG is officially formed. Extend the table as necessary.

Name

Organization

Ben Domenico

Unidata (NCAR)

Stefano Nativi,

Italian CNR-IMAA

Dominic Lowe

UK STFC

Roy Mendelssohn,

US NOAA

Luis Bermudez

US SURA

Rich Signell

USGS

Stefan Falke

Northrop Grumman

Jon Blower

University of Reading

Martin Desruisseaux

GEOMATYS

Chris Little

UK Met Office,

Marie Francoise Voidrot

Meteo France

Frederic Guilland,

Meteo France

 
  e. Convener(s) Ben Domenico, Unidata (NCAR)