# Policy Enforcement Gates for Data PRs

When spatial datasets, coordinate reference system definitions, and licensing metadata enter a repository through pull requests, manual review becomes both a bottleneck and a compliance liability. Policy enforcement gates solve this by automating the validation of spatial assets before they merge into production branches — verifying metadata completeness, license compatibility, and geometry integrity programmatically and consistently. This page is part of CI/CD Validation & Policy Enforcement for Spatial Data, where automated checkpoints replace ad-hoc review and apply organizational standards uniformly across distributed teams.

For GIS data managers, open-source maintainers, Python automation builders, and government agency tech teams, shifting these checks left into the pull request lifecycle reduces merge conflicts, prevents corrupted spatial indexes from reaching production, and produces auditable compliance records for regulatory reporting.

# Prerequisites

1. Version control platform — GitHub, GitLab, or Bitbucket with PR/MR webhook support and status check APIs.
2. Python 3.10+ — gdal==3.8.4, pyproj==3.6.1, jsonschema==4.22.0, lxml==5.2.2, requests==2.32.3.
3. Standardized metadata format — ISO 19115 XML, FGDC XML, or YAML/JSON templates aligned with your cataloging requirements (see ISO 19115 metadata template generation).
4. License registry — SPDX identifiers or an internal allowlist for commercial, open, and restricted-use spatial data licenses.
5. CI runner — Linux-based runner with pip access, network connectivity, optional GDAL/OGR binaries, and at least 4 GB RAM for raster geometry checks.
6. Policy rule definitions — version-controlled YAML or JSON configuration specifying required metadata fields, acceptable EPSG codes, and license compatibility matrices.
7. Environment variables — SPATIAL_POLICY_CONFIG (path to policy YAML), LICENSE_MATRIX_PATH (path to compatibility JSON), GH_STATUS_TOKEN or equivalent platform token for posting status check results.

# Concept & Spec Reference

A policy enforcement gate is a CI job that consumes a PR diff, applies a deterministic rule set, and returns a platform status check result. The three primary rule domains map to distinct technical obligations:

All three domains share the same enforcement contract: a blocking failure stops merge; a warning posts a PR comment requiring acknowledgement; a pass advances the PR to the merge queue. This three-tier model, combined with a structured override process, gives compliance teams the control they need without making every minor metadata gap a hard blocker.

For background on how metadata fields map to these standards, see automated metadata generation and schema mapping.

# Implementation Walkthrough

# Step 1 — Read the policy configuration

Load the version-controlled policy YAML so rules are auditable alongside the code.

import os
import yaml
from pathlib import Path

def load_policy(config_path: str | None = None) -> dict:
    """Load the spatial policy configuration from YAML."""
    path = Path(config_path or os.environ["SPATIAL_POLICY_CONFIG"])
    with path.open() as fh:
        return yaml.safe_load(fh)

# Example structure the YAML must contain:
# required_metadata_fields: [title, spatial_reference, license, contact_email, date_published]
# allowed_epsg_codes: [4326, 3857, 27700, 32631, 32632]
# license_allowlist: [CC-BY-4.0, ODbL-1.0, CC0-1.0, MIT]
# license_matrix_path: ./policy/license_matrix.json
# max_geometry_errors: 0
# link_timeout_seconds: 10

# Step 2 — Extract changed spatial files from the diff

Use the platform API (or git diff --name-only) to obtain only the paths touched by the PR, avoiding full-repository scans.

import subprocess
from pathlib import Path

SPATIAL_EXTENSIONS = {".gpkg", ".geojson", ".shp", ".tif", ".tiff", ".fgb"}
METADATA_EXTENSIONS = {".xml", ".yaml", ".yml", ".json"}

def get_changed_files(base_ref: str = "origin/main") -> tuple[list[Path], list[Path]]:
    """Return (spatial_files, metadata_files) changed versus base_ref."""
    result = subprocess.run(
        ["git", "diff", "--name-only", base_ref, "HEAD"],
        capture_output=True, text=True, check=True,
    )
    spatial, metadata = [], []
    for line in result.stdout.splitlines():
        p = Path(line.strip())
        if p.suffix.lower() in SPATIAL_EXTENSIONS:
            spatial.append(p)
        elif p.suffix.lower() in METADATA_EXTENSIONS:
            metadata.append(p)
    return spatial, metadata

# Step 3 — Validate metadata structure

Parse each metadata sidecar against a JSON Schema. The rationale: schema enforcement decouples field validation from business logic and makes error messages actionable.

import json
import jsonschema
from pathlib import Path

def validate_metadata_files(
    metadata_paths: list[Path],
    schema_path: Path,
    required_fields: list[str],
) -> list[dict]:
    """Validate metadata files; return a list of error dicts."""
    with schema_path.open() as fh:
        schema = json.load(fh)

    errors = []
    for path in metadata_paths:
        try:
            with path.open() as fh:
                doc = json.load(fh) if path.suffix == ".json" else {}
        except (json.JSONDecodeError, OSError) as exc:
            errors.append({"file": str(path), "type": "parse_error", "detail": str(exc)})
            continue

        try:
            jsonschema.validate(doc, schema)
        except jsonschema.ValidationError as exc:
            errors.append({"file": str(path), "type": "schema_error", "detail": exc.message})

        missing = [f for f in required_fields if not doc.get(f)]
        if missing:
            errors.append({"file": str(path), "type": "missing_fields", "detail": missing})

    return errors

# Step 4 — Resolve license compatibility

Extract SPDX identifiers and cross-reference them against the allowlist matrix. Raise a blocking error on incompatible or expired licenses.

import json
import re
import datetime
from pathlib import Path

SPDX_PATTERN = re.compile(r"SPDX-License-Identifier:\s*(\S+)")

def extract_spdx(text: str) -> str | None:
    """Extract the first SPDX identifier from file text."""
    m = SPDX_PATTERN.search(text)
    return m.group(1) if m else None

def check_license_compliance(
    metadata_paths: list[Path],
    allowlist: list[str],
    matrix_path: Path,
) -> list[dict]:
    """Return list of license violation dicts."""
    with matrix_path.open() as fh:
        matrix: dict[str, dict] = json.load(fh)

    today = datetime.date.today()
    violations = []

    for path in metadata_paths:
        try:
            content = path.read_text()
        except OSError:
            continue

        spdx_id = extract_spdx(content)
        if not spdx_id:
            violations.append({"file": str(path), "type": "missing_license"})
            continue

        if spdx_id not in allowlist:
            violations.append({"file": str(path), "type": "disallowed_license", "license": spdx_id})
            continue

        entry = matrix.get(spdx_id, {})
        expiry_str = entry.get("expiry")
        if expiry_str:
            expiry = datetime.date.fromisoformat(expiry_str)
            days_left = (expiry - today).days
            if days_left <= 0:
                violations.append({"file": str(path), "type": "expired_license", "license": spdx_id})
            elif days_left <= 30:
                violations.append({
                    "file": str(path), "type": "expiring_soon",
                    "license": spdx_id, "days_left": days_left,
                })

    return violations

# Step 5 — Run spatial integrity checks

Use osgeo.ogr to verify geometry validity, CRS declaration, and bounding-box sanity for each changed spatial file.

from osgeo import ogr, osr
ogr.UseExceptions()

def check_spatial_integrity(
    spatial_paths: list[Path],
    allowed_epsg_codes: list[int],
) -> list[dict]:
    """Return geometry/CRS error dicts for each file."""
    errors = []

    for path in spatial_paths:
        ds = ogr.Open(str(path))
        if ds is None:
            errors.append({"file": str(path), "type": "unreadable"})
            continue

        for layer_idx in range(ds.GetLayerCount()):
            layer = ds.GetLayerByIndex(layer_idx)
            srs = layer.GetSpatialRef()

            if srs is None:
                errors.append({"file": str(path), "type": "missing_crs", "layer": layer.GetName()})
            else:
                srs.AutoIdentifyEPSG()
                epsg = int(srs.GetAuthorityCode(None) or 0)
                if epsg and epsg not in allowed_epsg_codes:
                    errors.append({
                        "file": str(path), "type": "disallowed_crs",
                        "layer": layer.GetName(), "epsg": epsg,
                    })

            for feature in layer:
                geom = feature.GetGeometryRef()
                if geom is None or not geom.IsValid():
                    errors.append({
                        "file": str(path), "type": "invalid_geometry",
                        "layer": layer.GetName(), "fid": feature.GetFID(),
                    })

        ds = None  # close dataset

    return errors

# Step 6 — Aggregate results and post the status check

Combine all error lists into a single pass/warning/failure outcome and report it to the platform.

import os
import requests  # requests==2.32.3

GITHUB_API = "https://api.github.com"

def post_github_status(
    repo: str,
    sha: str,
    state: str,  # "success" | "failure" | "pending"
    description: str,
    context: str = "spatial-policy-gate",
) -> None:
    """Post a commit status check to GitHub."""
    token = os.environ["GH_STATUS_TOKEN"]
    url = f"{GITHUB_API}/repos/{repo}/statuses/{sha}"
    payload = {"state": state, "description": description[:140], "context": context}
    resp = requests.post(url, json=payload, headers={"Authorization": f"token {token}"})
    resp.raise_for_status()


def aggregate_and_report(
    metadata_errors: list[dict],
    license_violations: list[dict],
    spatial_errors: list[dict],
    repo: str,
    sha: str,
) -> int:
    """Return exit code: 0 = pass, 1 = blocking failure."""
    blocking_types = {
        "parse_error", "missing_fields", "schema_error",
        "missing_license", "disallowed_license", "expired_license",
        "missing_crs", "invalid_geometry", "unreadable",
    }
    warning_types = {"expiring_soon", "disallowed_crs"}

    all_issues = metadata_errors + license_violations + spatial_errors
    blocking = [i for i in all_issues if i["type"] in blocking_types]
    warnings = [i for i in all_issues if i["type"] in warning_types]

    if blocking:
        post_github_status(repo, sha, "failure", f"{len(blocking)} blocking issue(s) detected.")
        return 1
    if warnings:
        post_github_status(repo, sha, "success", f"Passed with {len(warnings)} warning(s) — review PR comments.")
        return 0
    post_github_status(repo, sha, "success", "All spatial policy checks passed.")
    return 0

# Validation & CI Integration

# GitHub Actions workflow

Wire the gate into a GitHub Actions workflow triggered on every pull request:

# .github/workflows/spatial-policy-gate.yml
name: Spatial Policy Gate
on:
  pull_request:
    paths:
      - "data/**/*.gpkg"
      - "data/**/*.geojson"
      - "data/**/*.shp"
      - "data/**/*.tif"
      - "metadata/**/*.yaml"
      - "metadata/**/*.json"

jobs:
  policy-gate:
    runs-on: ubuntu-22.04
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.10"

      - name: Cache pip
        uses: actions/cache@v4
        with:
          path: ~/.cache/pip
          key: ${{ runner.os }}-pip-${{ hashFiles('requirements-gate.txt') }}

      - name: Install dependencies
        run: pip install -r requirements-gate.txt

      - name: Run spatial policy gate
        env:
          SPATIAL_POLICY_CONFIG: policy/spatial_policy.yaml
          LICENSE_MATRIX_PATH: policy/license_matrix.json
          GH_STATUS_TOKEN: ${{ secrets.GH_STATUS_TOKEN }}
        run: python scripts/run_policy_gate.py

# Pre-commit hook for local validation

Running a lightweight subset of the gate locally shortens the feedback loop significantly and reduces CI runner consumption. Add this to .pre-commit-config.yaml:

repos:
  - repo: local
    hooks:
      - id: spatial-metadata-lint
        name: Spatial metadata lint
        language: python
        entry: python scripts/pre_commit_metadata_check.py
        types_or: [file]
        files: \.(gpkg|geojson|shp|tif|yaml|json)$
        pass_filenames: true
        additional_dependencies:
          - jsonschema==4.22.0
          - pyyaml==6.0.1

The pre-commit hook validates only staged files. Avoid running full geometry checks locally — limit the hook to metadata structure and license field presence, reserving OGR-based geometry validation for the CI runner where GDAL is fully provisioned.

For spatial schema linting patterns applied more broadly across the pipeline, see spatial data schema linting in CI.

# Derivative & Lineage Management

When a PR introduces a derived spatial layer — a reprojection, clip, join, or rasterization of an existing dataset — compliance obligations cascade:

• CRS transformation: a reprojection from EPSG:27700 to EPSG:4326 must be declared in the lineage metadata. The derived file’s metadata source_crs and target_crs fields must be populated, and the policy gate should verify these against the spatial_reference field of the source dataset.
• License inheritance: derivative works of ODbL-licensed data must carry an ODbL license on the output. Mixing an ODbL source with a CC-BY source produces an indeterminate compatibility state; the gate should route this to a human review queue rather than defaulting to allow.
• Geometry simplification: simplification algorithms (Douglas-Peucker, Visvalingam) alter topological relationships. Policy gates should flag layers whose processing_step metadata lists simplification and verify that a simplification_tolerance field is declared so downstream users can assess fitness for purpose.
• Clipped extents: bounding-box checks must account for intentionally restricted extents. The gate should compare the declared extent against the actual geometry envelope with a configurable tolerance (e.g., ±0.001 degrees), not a strict equality check.

For handling attribution obligations that travel with derived datasets, see geospatial data licensing compliance fundamentals.

# Handling Exceptions and Audit Trails

No automated gate is complete without a governed override path. An override requires three elements:

1. Explicit justification — the contributor provides a structured reason in a machine-readable format (e.g., a gate_override.yaml committed alongside the PR).
2. Secondary approval — a designated data steward or compliance officer approves the override via a required PR review.
3. Audit logging — every bypass is appended to an immutable log (append-only CI artifact or compliance database) with ISO 8601 timestamp, contributor user ID, reviewer user ID, and justification text.

Store audit logs as signed, append-only CI artifacts. The log entry schema should include at minimum: timestamp, pr_number, repository, bypassed_checks, justification, approver_id, and commit_sha. This enables organizations to demonstrate due diligence in regulatory audits by reconstructing the complete decision history for any spatial asset.

For strategies covering long-term retention of these audit artifacts alongside spatial data, see metadata artifact retention strategies.

# Pitfalls & Resolution Table

# Related

• CI/CD Validation & Policy Enforcement for Spatial Data — parent section covering the full pipeline
• Spatial data schema linting in CI — automated linting rules, schema versioning, and incremental validation
• Automated broken link and reference detection — detecting stale OGC service endpoints and metadata URLs
• Metadata artifact retention strategies — long-term storage and retrieval of compliance audit logs
• Automated metadata generation and schema mapping — generating ISO 19115 and DCAT-AP metadata programmatically