Geospatial Compliance Metadata Automation Hub

Writing a GitHub Actions Workflow for DCAT Validation

Writing a GitHub Actions workflow for DCAT validation means adding a pull_request-triggered job that parses each RDF catalog file with rdflib, validates it against a repository-stored DCAT-AP SHACL shapes graph using pyshacl, and blocks the merge whenever conforms is false. The result is a deterministic, offline gate that rejects non-compliant catalog metadata before it is published.

DCAT-AP is the application profile that governs how datasets are described in open data portals across the EU and, increasingly, North America; its spatial extension adds constraints on bounding geometry, reference systems, and licensing. Validating those constraints is a natural CI check, and it pairs with the Pre-commit Hooks for Spatial Metadata guide — the pre-commit hooks catch structural problems in the data files, while this workflow catches semantic problems in the catalog records that describe them. Both sit inside the CI/CD Validation & Policy Enforcement for Spatial Data framework. For how the DCAT-AP fields themselves are produced and mapped, see DCAT-AP spatial profile mapping.

Automated Python Implementation

The workflow triggers on pull requests that touch RDF catalog files or the SHACL shapes, installs pinned versions of rdflib and pyshacl, and runs a single validation script. Storing the shapes graph in the repository keeps validation deterministic and free of network calls at run time.

# .github/workflows/dcat-validate.yml
name: DCAT-AP SHACL Validation

on:
  pull_request:
    paths:
      - 'catalog/**/*.ttl'
      - 'catalog/**/*.rdf'
      - 'catalog/**/*.jsonld'
      - 'shapes/dcat-ap-shacl.ttl'
      - 'scripts/validate_dcat.py'

jobs:
  dcat-validate:
    runs-on: ubuntu-22.04
    timeout-minutes: 10
    steps:
      - name: Checkout repository
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'
          cache: 'pip'

      - name: Install RDF validation dependencies
        run: pip install "rdflib==7.0.0" "pyshacl==0.26.0"

      - name: Run DCAT-AP SHACL validation
        run: |
          python scripts/validate_dcat.py \
            --shapes shapes/dcat-ap-shacl.ttl \
            --catalog-dir catalog \
            --report reports/dcat_validation.txt

      - name: Upload validation report
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: dcat-validation-report
          path: reports/dcat_validation.txt
          retention-days: 14

The runner walks the catalog directory, parses each graph, and validates it against the shapes graph. pyshacl.validate returns a (conforms, results_graph, results_text) tuple; the script aggregates the text reports and exits non-zero if any file fails, which GitHub Actions interprets as a failed required check.

#!/usr/bin/env python3
"""scripts/validate_dcat.py — validate DCAT-AP catalog RDF against SHACL shapes.

Usage:
    python scripts/validate_dcat.py \
        --shapes shapes/dcat-ap-shacl.ttl \
        --catalog-dir catalog \
        --report reports/dcat_validation.txt
Exits 1 if any catalog graph fails SHACL validation.
"""
from __future__ import annotations

import argparse
import sys
from pathlib import Path

from pyshacl import validate
from rdflib import Graph

# Map file suffix to the rdflib parser format
FORMAT_BY_SUFFIX = {
    ".ttl": "turtle",
    ".rdf": "xml",
    ".xml": "xml",
    ".jsonld": "json-ld",
    ".nt": "nt",
}


def parse_format(path: Path) -> str:
    fmt = FORMAT_BY_SUFFIX.get(path.suffix.lower())
    if fmt is None:
        raise ValueError(f"{path}: unsupported RDF serialisation '{path.suffix}'")
    return fmt


def validate_catalog(data_path: Path, shapes_graph: Graph) -> tuple[bool, str]:
    """Validate a single catalog file. Returns (conforms, human-readable report)."""
    data_graph = Graph()
    try:
        data_graph.parse(str(data_path), format=parse_format(data_path))
    except Exception as exc:  # noqa: BLE001
        return False, f"{data_path}: RDF parse error — {exc}"

    conforms, _results_graph, results_text = validate(
        data_graph,
        shacl_graph=shapes_graph,
        inference="rdfs",
        abort_on_first=False,
        meta_shacl=False,
        advanced=True,
    )
    header = f"{data_path}: {'CONFORMS' if conforms else 'VIOLATIONS'}"
    return conforms, f"{header}\n{results_text.strip()}"


def main() -> int:
    parser = argparse.ArgumentParser(description="DCAT-AP SHACL validator")
    parser.add_argument("--shapes", required=True, type=Path)
    parser.add_argument("--catalog-dir", required=True, type=Path)
    parser.add_argument("--report", type=Path, default=Path("reports/dcat_validation.txt"))
    args = parser.parse_args()

    if not args.shapes.exists():
        print(f"ERROR: shapes graph not found: {args.shapes}", file=sys.stderr)
        return 1

    shapes_graph = Graph()
    shapes_graph.parse(str(args.shapes), format="turtle")

    catalog_files = sorted(
        p for p in args.catalog_dir.rglob("*")
        if p.suffix.lower() in FORMAT_BY_SUFFIX
    )
    if not catalog_files:
        print("No catalog RDF files found — skipping validation.")
        return 0

    reports: list[str] = []
    failed = 0
    for path in catalog_files:
        conforms, report = validate_catalog(path, shapes_graph)
        reports.append(report)
        if not conforms:
            failed += 1

    args.report.parent.mkdir(parents=True, exist_ok=True)
    args.report.write_text("\n\n".join(reports))

    print("\n\n".join(reports))
    summary = f"\n{failed} of {len(catalog_files)} catalog file(s) failed DCAT-AP validation."
    print(summary)
    return 1 if failed else 0


if __name__ == "__main__":
    sys.exit(main())

The inference="rdfs" setting lets pyshacl apply RDFS entailment before validation, so shapes that target dcat:Dataset still match a node typed only as a subclass. advanced=True enables SHACL-AF features such as sh:SPARQLConstraint, which the DCAT-AP shapes use for cross-property rules like requiring dct:spatial when a dct:Location bounding box is present. Setting abort_on_first=False means one run reports every violation, not just the first.

Validation & pipeline integration

Run the validator locally against the same pinned versions the workflow uses before opening a pull request:

# Match the workflow's pinned dependency versions
pip install "rdflib==7.0.0" "pyshacl==0.26.0"

# Validate the whole catalog directory
python scripts/validate_dcat.py \
    --shapes shapes/dcat-ap-shacl.ttl \
    --catalog-dir catalog \
    --report reports/dcat_validation.txt

# Validate a single record while debugging a specific failure
python -c "from rdflib import Graph; g = Graph(); g.parse('catalog/roads.ttl', format='turtle'); print(len(g))"

Lock the behaviour behind a pytest regression so a shapes edit that silently stops catching a violation is itself caught:

# tests/test_dcat_validation.py
from pathlib import Path

from rdflib import Graph

from scripts.validate_dcat import validate_catalog

SHAPES = Graph().parse("shapes/dcat-ap-shacl.ttl", format="turtle")


def test_missing_license_fails(tmp_path: Path):
    ttl = """
    @prefix dcat: <http://www.w3.org/ns/dcat#> .
    @prefix dct: <http://purl.org/dc/terms/> .
    <http://example.org/ds/1> a dcat:Dataset ;
        dct:title "Roads" .
    """
    path = tmp_path / "no_license.ttl"
    path.write_text(ttl)
    conforms, report = validate_catalog(path, SHAPES)
    assert conforms is False
    assert "VIOLATIONS" in report


def test_complete_record_conforms(tmp_path: Path):
    ttl = """
    @prefix dcat: <http://www.w3.org/ns/dcat#> .
    @prefix dct: <http://purl.org/dc/terms/> .
    <http://example.org/ds/2> a dcat:Dataset ;
        dct:title "Parcels" ;
        dct:description "Cadastral parcels for the district." ;
        dct:license <http://publications.europa.eu/resource/authority/licence/CC_BY_4_0> .
    """
    path = tmp_path / "ok.ttl"
    path.write_text(ttl)
    conforms, _report = validate_catalog(path, SHAPES)
    assert conforms is True

Once the job is green, promote dcat-validate to a required status check in Settings → Branches → Branch protection rules so it wires into the policy enforcement gates for data PRs and blocks non-conforming catalog records at the SCM layer.

Long-term compliance best practices

  • Vendor the SHACL shapes into the repository. Fetching shapes from a remote URL at run time makes validation non-deterministic and network-dependent; a committed shapes/dcat-ap-shacl.ttl guarantees every run validates against the exact profile revision you reviewed.
  • Pin rdflib and pyshacl to exact versions. Both libraries evolve their SHACL support between releases; an unpinned upgrade can change which violations are reported, turning a passing branch red for no data reason.
  • Separate parse errors from conformance failures. A malformed Turtle file and a well-formed but non-conforming record are different defects with different fixes — the runner reports them distinctly so contributors know whether to fix syntax or content.
  • Keep RDFS inference on only when your shapes need it. Inference costs time and can mask missing explicit types; enable it deliberately and document why, rather than leaving it on by habit.
  • Publish the validation report as a build artifact. Uploading reports/dcat_validation.txt on every run — including passes — gives auditors a durable record of which catalog records conformed at merge time.
  • Align the shapes with your field mapping. When the DCAT-AP spatial profile mapping that generates the records changes, update the shapes in the same pull request so producer and validator never drift apart.