Geospatial Compliance Metadata Automation Hub

Enforcing License Fields in Data Pull Requests

Enforce license fields in a data pull request by scanning every changed dataset’s metadata for a required license identifier, confirming that identifier is recognized against an SPDX-based allowlist, and exiting non-zero when any dataset is missing a license or declares an unknown one — which blocks the merge until the licensing is corrected. A dataset without a clear, machine-readable license is a redistribution liability: nobody downstream can safely determine whether they may publish, sell, or combine it.

This guide implements one specific rule within Policy Enforcement Gates for Data PRs, the parent guide that aggregates metadata, license, and geometry checks into a single merge decision. It belongs to the broader CI/CD Validation & Policy Enforcement for Spatial Data framework. The scope here is deliberately tight: presence and recognizability of the license field itself. Deeper questions — whether two licenses may be combined, or which license to pick for a new dataset — are handled by commercial EULA compliance tracking and open data license selection and comparison respectively.

Automated Python Implementation

The script below is self-contained and depends only on the Python standard library plus PyYAML for parsing YAML sidecars. It reads an allowlist of accepted SPDX identifiers, inspects each metadata record passed on the command line, and reports every record that is missing a license field or declares one the allowlist does not recognize. It exits 1 on any violation so a CI job fails the check.

#!/usr/bin/env python3
"""Enforce a required, recognized license field on changed datasets.

Usage:
    python enforce_license_fields.py \
        --allowlist policy/license_allowlist.txt \
        metadata/parcels.yaml metadata/zoning.json

The allowlist file holds one SPDX identifier per line; blank lines and
lines beginning with '#' are ignored.

Exit codes:
    0 — every record declares a recognized license
    1 — at least one record is missing or declares an unknown license
    2 — a usage or file error occurred
"""
import argparse
import json
import sys
from pathlib import Path

import yaml

# Metadata keys that may hold the license identifier, in priority order.
LICENSE_KEYS = ("license", "license_id", "spdx_license", "rights")


def load_allowlist(path: Path) -> set[str]:
    """Read recognized SPDX identifiers from a newline-delimited file."""
    entries: set[str] = set()
    for line in path.read_text(encoding="utf-8").splitlines():
        token = line.strip()
        if token and not token.startswith("#"):
            entries.add(token)
    return entries


def read_metadata(path: Path) -> dict:
    """Parse a YAML or JSON metadata sidecar into a dict."""
    text = path.read_text(encoding="utf-8")
    if path.suffix.lower() in {".yaml", ".yml"}:
        data = yaml.safe_load(text)
    elif path.suffix.lower() == ".json":
        data = json.loads(text)
    else:
        raise ValueError(f"unsupported metadata format: {path.suffix}")
    if not isinstance(data, dict):
        raise ValueError(f"{path}: metadata root is not a mapping")
    return data


def extract_license(meta: dict) -> str | None:
    """Return the first non-empty license value among the accepted keys."""
    for key in LICENSE_KEYS:
        value = meta.get(key)
        if isinstance(value, str) and value.strip():
            return value.strip()
    return None


def check_record(path: Path, allowlist: set[str]) -> dict | None:
    """Return a violation dict for one record, or None if it passes."""
    try:
        meta = read_metadata(path)
    except (ValueError, yaml.YAMLError, json.JSONDecodeError) as exc:
        return {"file": str(path), "type": "unparseable", "detail": str(exc)}

    license_id = extract_license(meta)
    if license_id is None:
        return {"file": str(path), "type": "missing_license",
                "detail": f"no license found under keys {LICENSE_KEYS}"}
    if license_id not in allowlist:
        return {"file": str(path), "type": "unrecognized_license",
                "detail": f"'{license_id}' is not in the allowlist"}
    return None


def main(argv: list[str]) -> int:
    parser = argparse.ArgumentParser(description="Enforce license fields on datasets.")
    parser.add_argument("--allowlist", required=True, type=Path,
                        help="File of accepted SPDX identifiers, one per line.")
    parser.add_argument("records", nargs="*", type=Path,
                        help="Metadata sidecar files to check.")
    args = parser.parse_args(argv[1:])

    if not args.allowlist.exists():
        print(f"ERROR allowlist not found: {args.allowlist}")
        return 2
    if not args.records:
        print("No metadata records to check — nothing changed.")
        return 0

    allowlist = load_allowlist(args.allowlist)
    violations = [v for p in args.records if (v := check_record(p, allowlist))]

    for v in violations:
        print(f"FAIL {v['file']}: {v['type']} — {v['detail']}")
    if violations:
        print(f"\n{len(violations)} dataset(s) failed the license gate.")
        return 1

    print(f"All {len(args.records)} dataset(s) declare a recognized license.")
    return 0


if __name__ == "__main__":
    sys.exit(main(sys.argv))

The check separates three failure modes so error messages are actionable: an unparseable sidecar, a missing license field, and a recognized-but-not-approved identifier. Accepting several candidate keys (license, license_id, spdx_license, rights) lets the gate work across ISO 19115-derived records and lighter YAML templates without forcing every team onto one field name.

Validation and pipeline integration

Run the gate locally against a good record and a bad one before wiring it into CI. Keeping the allowlist in version control means the accepted set is auditable and changes go through review like any other code:

# A minimal allowlist under version control
cat > policy/license_allowlist.txt <<'EOF'
# Approved data licenses (SPDX identifiers)
CC-BY-4.0
CC-BY-SA-4.0
CC0-1.0
ODbL-1.0
PDDL-1.0
EOF

# A compliant record and a non-compliant one
printf 'title: Parcels\nlicense: CC-BY-4.0\n'   > metadata/good.yaml
printf 'title: Zoning\ncontact: gis@example.gov\n' > metadata/bad.yaml

python enforce_license_fields.py \
    --allowlist policy/license_allowlist.txt \
    metadata/good.yaml metadata/bad.yaml
echo "exit code: $?"   # expect 1 — bad.yaml has no license

Add a pytest case so the rule cannot regress:

from pathlib import Path
from enforce_license_fields import check_record

ALLOW = {"CC-BY-4.0", "ODbL-1.0"}


def test_missing_license_flagged(tmp_path):
    rec = tmp_path / "m.yaml"
    rec.write_text("title: X\n", encoding="utf-8")
    result = check_record(rec, ALLOW)
    assert result is not None and result["type"] == "missing_license"


def test_recognized_license_passes(tmp_path):
    rec = tmp_path / "m.yaml"
    rec.write_text("title: X\nlicense: ODbL-1.0\n", encoding="utf-8")
    assert check_record(rec, ALLOW) is None

Finally, trigger the gate from GitHub Actions on any change to a metadata record, passing only the changed files so the job scales with the size of the diff:

name: License Field Gate
on:
  pull_request:
    paths:
      - "metadata/**/*.yaml"
      - "metadata/**/*.yml"
      - "metadata/**/*.json"

jobs:
  license-gate:
    runs-on: ubuntu-24.04
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
      - uses: actions/setup-python@v5
        with:
          python-version: "3.11"
          cache: "pip"
      - name: Install
        run: pip install pyyaml==6.0.1
      - name: Enforce license fields on changed records
        run: |
          git diff --name-only --diff-filter=ACMRT origin/${{ github.base_ref }} HEAD \
            | grep -E 'metadata/.*\.(ya?ml|json)$' \
            | xargs -r python scripts/enforce_license_fields.py \
                --allowlist policy/license_allowlist.txt

Promote license-gate to a required status check under branch protection so a missing license blocks the merge at the source-control layer, consistent with the enforcement model in the parent gates guide.

Long-term compliance best practices

  • Version-control the allowlist and require review to change it. Adding an identifier is a policy decision. Keeping the allowlist in the repository means every addition is reviewed, dated, and attributable — the audit trail comes for free.
  • Standardize on SPDX identifiers, never free text. A stable token like ODbL-1.0 is machine-checkable and lets downstream compatibility tooling reason about the license. Reject values such as “open” or “see website” outright.
  • Fail closed on unknown licenses. Treat an unrecognized identifier as a blocking failure routed through a governed override, not a warning. A permissive default lets unvetted terms accumulate silently until an audit forces a costly cleanup.
  • Check derivatives inherit a valid license too. A reprojected or clipped output is a new dataset and needs its own recognized license field; run the same gate on generated products, informed by open data license selection and comparison when choosing the derivative’s terms.
  • Separate presence from compatibility. This gate proves a license exists and is recognized. Whether an ODbL source may be combined with a commercial layer is a distinct question handled by commercial EULA compliance tracking; keep the two checks modular.
  • Record every override with justification and approver. When a genuinely new license is admitted, log the identifier, the PR, the steward who approved it, and a timestamp so the exception is reconstructable during a compliance review.