Standard Compliance: Run Tests

Version 12 includes a built-in SPARQL 1.1 query evaluator that allows querying in-memory RDF (Resource Description Framework) data. The RDF Tests Community Group maintains a repository of tests for RDF and SPARQL. Test manifests, which list the available tests and their metadata like type (evaluation test, syntax test, ...) and resources required to run them, are serialized in the "Turtle" format.

This example demonstrates how to extract test metadata from a manifest using SPARQL and how to run those tests.

Start by downloading and extracting the repository containing W3C tests for RDF and SPARQL.

The test suite contains many categories of tests: RDF import, SPARQL query parsing and evaluation, entailment, …. Choose the tests concerning aggregates.

The manifest file contains a list of tests and metadata for each test like the input data, query to be executed and expected result. Snippet allows you to peek into a file, so have a look. First, you see the vocabulary used by the test manifest.

Here is the start of the list of tests.

And here is an example of a test specification.

Use SPARQL to extract information about each test. Load GraphStore and specify vocabulary helper functions.

Look at which test types there are, and how many of each.

Run the evaluation tests: First, retrieve all evaluation tests and their metadata.

Here is the first one.

For each test, import the query and apply it to the test data. Then compare the result to the expected result.