WFS 1.0.0 Testing Instructions

Preparation

  1. Verify that your service is running on a supported port number. The port numbers the engine supports are listed in the release notes, accessible from the home page of the engine.

    Note: The port number is the number following the hostname in the URL, if there is one, or 80 by default. For example, in www.somehost.com:8080/wms, the port number is 8080. In www.anotherhost.com/wms, it is 80.

  2. Load the test dataset into your service. Test datasets can be downloaded from the CITE portal. The engine may be able to run some of the tests even if you skip this step, but most of the tests depend on the dataset, so a much larger set of tests will be executed when the dataset is properly implemented. The service will not pass certification testing if you skip this step.

Running Tests

  1. From the CITE Test Engine home page, click on the “Start Testing” link. You will be prompted to enter the username and password you created in step 1 above, if you have not done so earlier. The “Test Sessions” page will appear.

    Warning: The engine uses HTTP Basic authentication. Usernames and passwords are not encrypted.

  2. Create a Test Session.
    • From the CITE Test Engine home page, click on the “Manage Test Sessions” link. You will be prompted to enter the username and password you created in step 1 above, if you have not done so earlier. The “Test Session Management Menu” page will appear.
    • Select the “Create a new session” link. The “Select test suite” page will appear.
    • You will see four (4) drop-down menus. In the first, "Organization", choose "OGC". In the second, "Standard", select the service that you want to test (i.e. WFS or WMS). In the third and fourth menus, select the appropriate values. Finally, enter a session description if desired, such as "WFS 1.0.0 working" and press the “Start a new test session” button at the bottom of the page. A second “Customize Test Session” page will appear, listing options available for the service selected.
    • Fill in the options and press the “Add Capabilities Values button at the bottom of the page. The engine will load the capabilities document to determine which options the engine supports and display a new page listing the values.
    • Press the “Create Test Session” button at the bottom of the page. You will be returned to the “Test Session Management Menu” page. The test session you created should show up in a table listing existing test sessions.

  3. Execute the Tests. From the “Test Session Management Menu” page, select “Execute Tests” to execute all the tests applicable to the session. Alternatively, to execute a smaller set of tests select “View Test Results” to view the tests in the session, scroll down to the test you want to execute, and select its “Execute Test” link. The engine will execute the selected test, and continue by automatically executing the next UNTESTED or FAILED test in the list. When the engine has completed the tests you are interested in, you can stop it by pressing the “Stop” button. If you want to rerun all of the tests, including the ones that PASSED, return to the “Test Session Management Menu” page, select “Manage Session”, and select “Clear the Test Session results”.

Evaluating the Results

After executing the tests, return to the “Test Session Management Menu” page and select the “View Test Results” link. At the top of this page, there will be a table summarizing the number of tests that have Passed, Failed, or are yet to be tested. The rest of the page contains a table listing each of the tests applicable to the session along with its result (Passed, Failed, or Untested).

Examine Test Page

Information about a test is always available from the “Test Results” page by clicking on the Test ID of the test you want to examine to bring up the “Examine Test” page. The page has several sections:

  1. Main Information. Shows general information about the test. Of particular interest in this section are:
    • Assertion - States the behavior to be tested.
    • Specification Notes – Details how the behavior is tested.
    • Specification Reference – Links back to the section in the specification where the behavior is defined.
    • Execution Type – Specifies whether the test is manual (requires user interaction), or auto (executed automatically).
  2. Expected Results. Shows what is required to happen in order for the test to pass. There are several possible results types:
    • URL Results - Manual tests generally contain a “URL” result, meaning that the result is read from the URL returned from the browser as a result of user interaction. In this case, the values of the key-value pairs that must be present are shown.
    • Xpath results - Automatic tests generally contain an “Xpath” or “custom” result. For an “Xpath” result, an Xpath expression is evaluated against the results of the last request (see Requests below). The Xpath expression and its expected value are listed. If the value of the “Check” column is “headers” instead of “body”, the Xpath expression is evaluated against an XML representation of the HTTP headers returned by the request instead of the content returned by the request.
    • Custom results - A “custom” result works the same way as an “Xpath” result, except that the response is transformed into XML using a custom Java class before the expression is evaluated. The name of the Java class and any parameters passed to the class are listed.
  3. Variables set within this test. If there are variables set in response to any of the requests, there will be a section in the page that shows them. The variables may be used in the Xpath expression in the results section by enclosing the variable name in double brackets. For example, expression “‘[[VAR1]]’ = ‘test’” returns true if there is a variable named VAR1 whose value is “test”.
  4. Test Source Method. Shows the HTML that is sent to the browser to run the test. This may contain useful information for manual tests, but is generally uninteresting for automatic tests since it just redirects the browser to the URL that executes the automatic test.

  5. Requests. Shows the request(s) sent to the server during the test. If the request method is “get”, the URL is displayed. If the request method is post, the content posted to the server is also displayed. For each request, the HTML returned to the user is also displayed. This is not the server’s response to the request (view the execution log to see that), but HTML sent back to the browser after the request is made. It generally just redirects the browser to the URL to process the next request (if there are any), or to move to the next test. If “Validate against Schema /DTD” is set to true, then the response from the server will be validated to determine whether the test passed or failed. Otherwise, if “Validate Content” is set to true, the response will be evaluated as detailed in the Expected Results section above.

Execution Log Page

Once a test has been run, there will be an execution log created for it. You can view the log by returning to the “Test Results” page and selecting the “View Test Log” option next to the test you want to see.

The log lists information obtained by running the test. It shows both the HTTP headers and the body of the response received from the server for each request. If any variables are set by the test, their values are listed. If the test failed, there will be an Errors section showing the result value that was expected, the actual value received, any other informative messages, if available. You can examine the test from this page by clicking on the “Test Path” link to see more about the expected result value or other test info.

More Info

Additional Information can be found in the “Quick Start User Guide” link on the home page of the test engine.