Conformance & Interoperability Test & Evaluation, Phase 1
As customers in the geospatial industry prepare to modernize their
systems based on OGC Web Services, they will require proof of products'
conformance to OpenGIS® Specifications in order to achieve
interoperability. Validating conformance to an OpenGIS® Specification
means verifying that a software product has implemented the
specification correctly. Validation involves testing the software
interface for response and behavior outlined in the specification. The
Conformance and Interoperability Test and Evaluation (CITE) Initiative
has provided the geospatial industry (customers and vendors) with a
methodology and tools for testing products' conformance to OGC Web
Services specifications and for determining whether products that
conform are interoperable with one another. Very importantly, CITE also
produced conformance tests for the OpenGIS Web Map Service
Specification (WMS) and the OpenGIS Web Feature Service Specification
(WFS). In addition, CITE produced a conformance engine (a platform for
developing tests), a GML validator, scripts and guidelines, and a CITE
Portal with links to Websites that implement the official OGC reference
implementations of WMS and WFS.
For more information on the CITE Initiative, please contact Kurt Buehler (kbuehler@opengeospatial.org).
1). 9/9/2002 - Questions brought up during CTIE Subcommittee Meetings in Noordwijk
CITE RFQ content. It is also included in the Zip file with the
complete RFQ package.
discussed the CITE RFQ during the Technical Committee meeting
in Noordwijk, the Netherlands (9/9/02). The discussion resulted
in the need to clarify several items as follows.
- "How will the development of the test engine be affected
by the planning study?"
The Planning and Feasibility Study is mostly programmatic
and not, necessarily, technical in nature. The schedule is
designed such that the Planning Study draft and Initial Conformance
Engine design are due coincidentally. This will enable technical
issues unearthed during the Planning and Feasibility Study
to be addressed by the Conformance Engine developers.
- "Should proposals include discussion about data sets
for testing?"
Well-formed proposals for the Conformance Test Scripts and
Guidelines should address the issue of test data. In other
words, preliminary recommendations about data sets should
be provided.
- "Will OGC charge Trademark Licensing Fees to people providing
Reference Implementations under the CITE Initiative?"
OGC will waive Trademark Licensing Fees for Reference Implementations
that are accepted for use under the CITE initiative that also
pass conformance tests once those are adopted by the OGC for
official use.
- "Will travel costs be covered for those responding to
tasks requiring software installation?"
Please include any expected travel costs in your proposal;
we expect to negotiate with you on this matter in order to
avoid these costs, where possible.
2). 30 September 2002 - Question:
The specification [RFQ] is vague regarding the requirements
for testing the WFS.
It also implies that a WFS tester must also be capable of testing
a WMS.
This does not seem reasonable to us, although we appreciate that
a general test harness can be useful, and our submission would be
extensible to testing WMS also.
We refer to the paragraph in Annex B which reads:
"At a minimum, Guidelines and Test Scripts will be developed
for the Web Mapping Server (WMS) and Web Feature Server (WFS) implementation
specifications. OGC will also investigate the feasibility of implementing
an online Geography Markup Language (GML) validation tool, which
would be the method for testing conformance of GML documents generated
by the WFS."
Specifically then:
Must any WFS test component also be capable of testing
a WMS?
Is the testing of a WFS with respect to the validity of
all responses (Capabilities, DescribeFeatures, GetFeature etc.)
a Tier 1 activity in this RFQ? Please clarify the last sentence
in the paragraph above. Note that GML or GML-related content is
contained in many WFS responses (e.g. GetCapabilities, DescribeFeature,
GetFeature). Validating the GML from a GetFeature request is only
one part of the WFS test.
Clarification:
Proposal responses do not have to include the development of test
scripts for both WMS and WFS. The response should include a description
of how individual test scripts will be executed with a general test
engine.
Conformance tests for WMS, WFS and GML are all Tier 1 requirements.
The following statement:
"OGC will also investigate the feasibility of implementing
an online Geography Markup Language (GML) validation tool, which
would be the method for testing conformance of GML documents generated
by the WFS" should be revised to read "Proposal responses
should indicate how the Geography Markup Language (GML) validation
tool developed in this initiative will be the utilized to test conformance
of GML documents generated by
the WFS."
3). 1 October 2002 - Question:
It
is clear in the RFQ that OGC seeks test tools for WMS and WFS servers,
but it is not clear whether test tools for client applications are also
sought. Please clarify.
Clarification:
It is not the specific intent of this RFQ to request such
tools.
4). 1 October 2002 - Question:
WBS 8.3 requires a performance test. Against what criteria will performance be assessed?
Clarification:
You must be asking about WBS 8.4.3. This requirement
establishes that you will provide an individual to participate in
performance tests. The criteria are TBD, however, the primary abstract
criteria is:
- Does the implementation properly conformance test the
interfaces, operations, encodings, and messages that are within the WMS,
WFS, GML and other OGC web-based specifications.
OGC will endeavor to develop more detailed, mutually agreeable
performance criteria before the testing occurs. NOTE: Performance
Testing is not meant to indicate the execution speed of the
implementation.
5). 1 October 2002 - Question:
WBS 9.3 allows delivery of the conformance test system to be by remote
access to the developer's site. WBS 12.2.2 requires delivery of source
code. Please clarify the requirement.
Clarification:
You must be asking about WBS 9.1 and 12.2.2. The mention of source code in 12.2.2 is intended to refer to the test script
specifically and not the test engine, necessarily.
Updated: 2009-11-21 02:17:13 EST
