# CI/CD Validation & Policy Enforcement for Spatial Data

Modern geospatial operations have outgrown manual review cycles. As agencies, open-source projects, and enterprise GIS teams scale their spatial data pipelines, the risk of publishing malformed geometries, non-compliant licenses, or incomplete metadata grows in proportion to the number of contributors and datasets under management. Treating spatial validation as a manual, ad-hoc activity introduces compliance debt that compounds silently until a regulatory audit, a broken data feed, or a public-facing map error forces a costly remediation sprint.

Implementing automated validation and policy enforcement transforms those ad-hoc quality checks into deterministic, auditable guardrails that intercept problems before data reaches production. This domain covers the full stack: version-controlled data storage, structural validation, metadata artifact retention strategies, policy evaluation gates, and immutable audit log generation. It is designed for GIS data managers, open-source maintainers, Python automation builders, and government and agency technical teams seeking reproducible, standards-aligned data delivery workflows.

# Core Concepts & Standards

Spatial data pipelines require a fundamentally different validation model than software compilation. Geospatial assets carry coordinate-system dependencies, implicit topological relationships, and regulatory obligations that cannot be surfaced through syntax checking alone. The following standards and frameworks define the compliance space:

• OGC Simple Features (ISO 19125) — The normative reference for geometry validity. Defines what constitutes a valid polygon, linestring, and multipart feature. Every topology check in a spatial CI pipeline should trace back to this specification. See spatial data schema linting in CI for implementation patterns.

• ISO 19115 Geographic Information Metadata — Mandates the metadata fields required for dataset discovery, provenance, and rights documentation in government and regulated contexts. Automated pipelines that extract and validate these fields are covered in the ISO 19115 metadata template generation workflow.

• FGDC Content Standard for Digital Geospatial Metadata — The US Federal Geographic Data Committee’s metadata standard, widely required for federal agency data submissions. FGDC-to-ISO 19115 conversion pipelines handle automated cross-walking between these two formats.

• DCAT-AP Spatial Profile — The W3C Data Catalog Vocabulary Application Profile used across EU member states and increasingly adopted in North American open data portals. DCAT-AP spatial profile mapping covers field-level translation from ISO 19115 and FGDC sources.

• SPDX License Expressions — Machine-readable license identifiers (e.g., ODbL-1.0, CC-BY-4.0) used to declare and enforce data licensing constraints inside validation pipelines. Validated alongside metadata fields to prevent unlicensed or incompatibly licensed data from entering production catalogs.

• STAC (SpatioTemporal Asset Catalog) — An emerging JSON-based specification for cloud-native spatial asset catalogs. STAC item validation can be integrated as a CI gate to ensure API-readable catalog entries conform before publication.

• Policy-as-Code — The practice of encoding organizational governance rules in YAML, JSON, or Python that evaluation engines — such as Open Policy Agent (OPA) or custom Python evaluators — execute against structured validation output. Covered in detail at policy enforcement gates for data PRs.

# Compliance Obligations & Risk Surface

When spatial validation is absent or inconsistently applied, specific failure modes recur across team types and data domains.

Schema drift occurs when contributors add or rename attribute columns without updating downstream consumers. In regulated contexts — federal land records, census boundary files, emergency management layers — schema drift invalidates pre-certified data contracts and can trigger re-certification requirements. A single missing effective_date column in a zoning dataset can invalidate permits issued against that layer.

Coordinate reference system (CRS) contamination happens when datasets recorded in a local projected CRS (e.g., EPSG:26917) are published without an explicit CRS declaration, or when automated reprojection silently applies the wrong transformation. Downstream users who assume WGS84 (EPSG:4326) will produce positional errors measured in hundreds of metres — sufficient to misplace infrastructure features, violate buffer zone regulations, or corrupt boundary analysis.

Broken external references in metadata — service URLs, linked datasets, external WMS endpoints — degrade data usability and can constitute compliance violations when metadata records are submitted to national or international registries. Automated broken link and reference detection prevents this class of defect from reaching production.

License contamination arises when a dataset ingested under a share-alike license (ODbL, CC-BY-SA) is combined with a dataset that carries a more restrictive commercial EULA, producing a derivative whose redistribution terms are undefined or legally conflicted. Automated commercial EULA compliance tracking and geospatial risk scoring frameworks quantify this risk before merges.

Audit trail gaps expose organizations during FOIA requests, legal discovery, or inter-agency data exchange reviews. If no machine-readable record exists of which validation rules ran, which version of the data was checked, and who approved an exception, the organization cannot demonstrate due diligence. Every CI run must produce a signed, timestamped report stored in immutable artifact storage.

Topology violations — self-intersecting polygons, unclosed rings, coordinate precision loss — are frequently introduced when contributors use desktop GIS tools with automatic snap-and-fix features that silently alter geometry. These fixes may be appropriate for rendering but are inappropriate for legal boundary or land use datasets where geometric precision has regulatory significance.

# Engineering Integration Patterns

A production-ready geospatial validation pipeline assembles five sequential stages, each producing machine-readable output that feeds the next.

Stage 1 — Version-controlled data checkout. Changed spatial files are retrieved from Git LFS, DVC, or a dedicated data registry. Only modified files are validated per run to keep execution time proportional to change scope.

Stage 2 — Schema linting. ogrinfo, pyogrio, and jsonschema inspect file structure against a YAML manifest that declares required columns, accepted CRS codes, geometry types, and file format constraints. The linting stage fails fast: a missing required column terminates validation before expensive topology checks run.

Stage 3 — Metadata extraction and compliance. lxml, owslib, or custom parsers extract ISO 19115 or FGDC metadata from sidecar files or embedded records. Extracted fields are validated against required-field lists and cross-referenced against the SPDX license allowlist. Broken service references are probed via HTTP. This stage produces a structured JSON compliance report.

Stage 4 — Topology and geometry validation. shapely, pyproj, and rasterio execute geometry validity checks aligned to OGC Simple Features. Raster pipelines validate band counts, pixel value ranges, and alignment grids. Coordinate transformation smoke tests verify that projection round-trips stay within defined tolerance thresholds.

Stage 5 — Policy gate. A policy evaluator ingests the compliance JSON from stages 2–4 and applies organizational rule sets. Hard violations (missing license, invalid geometry, CRS undeclared) block the merge automatically. Soft violations (file size above threshold, metadata completeness below 90%) route the PR to a designated steward via CODEOWNERS review requests.

The following overview script illustrates how these stages wire together in a single Python runner:

import json
import pathlib
import sys

import geopandas as gpd
import pyogrio
from shapely.validation import explain_validity

REQUIRED_COLUMNS = {"zone_type", "effective_date", "data_license"}
ALLOWED_CRS = {"EPSG:26917", "EPSG:4326", "EPSG:4269"}
SPDX_ALLOWLIST = {"ODbL-1.0", "CC-BY-4.0", "CC-BY-SA-4.0", "PDDL-1.0"}


def lint_schema(path: pathlib.Path) -> list[str]:
    """Return list of schema violation messages, empty on pass."""
    errors = []
    info = pyogrio.read_info(str(path))
    crs_auth = info.get("crs_wkt", "")
    # Simplified CRS check: look for EPSG code in WKT
    if not any(code in crs_auth for code in ALLOWED_CRS):
        errors.append(f"CRS not in allowlist: {crs_auth[:60]!r}")
    gdf = gpd.read_file(path)
    missing = REQUIRED_COLUMNS - set(gdf.columns)
    if missing:
        errors.append(f"Missing required columns: {sorted(missing)}")
    return errors


def check_geometry(path: pathlib.Path) -> list[str]:
    """Return topology violation messages."""
    errors = []
    gdf = gpd.read_file(path)
    for idx, geom in enumerate(gdf.geometry):
        if geom is None or geom.is_empty:
            errors.append(f"Feature {idx}: null or empty geometry")
            continue
        if not geom.is_valid:
            errors.append(f"Feature {idx}: {explain_validity(geom)}")
    return errors


def check_license(path: pathlib.Path) -> list[str]:
    """Validate data_license column values against SPDX allowlist."""
    errors = []
    gdf = gpd.read_file(path)
    if "data_license" not in gdf.columns:
        return ["data_license column absent — cannot verify licensing"]
    invalid = set(gdf["data_license"].dropna().unique()) - SPDX_ALLOWLIST
    if invalid:
        errors.append(f"Unlicensed or non-approved SPDX identifiers: {invalid}")
    return errors


def run_validation(changed_files: list[pathlib.Path]) -> dict:
    report = {"files": {}, "overall": "pass"}
    for path in changed_files:
        file_result: dict[str, list[str]] = {
            "schema": lint_schema(path),
            "geometry": check_geometry(path),
            "license": check_license(path),
        }
        all_errors = file_result["schema"] + file_result["geometry"] + file_result["license"]
        file_result["status"] = "fail" if all_errors else "pass"
        if file_result["status"] == "fail":
            report["overall"] = "fail"
        report["files"][str(path)] = file_result
    return report


if __name__ == "__main__":
    targets = [pathlib.Path(p) for p in sys.argv[1:]]
    result = run_validation(targets)
    print(json.dumps(result, indent=2))
    sys.exit(0 if result["overall"] == "pass" else 1)

This script is intentionally modular: each function maps directly to a pipeline stage, making it straightforward to extend with additional checks or swap in alternative libraries without restructuring the runner.

# Metadata Schema Requirements

Every spatial dataset entering a governed pipeline must carry a verifiable metadata record. The following table maps required fields to the applicable standards and the validation approach used in automated pipelines:

Pipelines should validate these fields at PR time using lxml (for XML metadata sidecar files), jsonschema (for JSON/STAC metadata), or owslib (for WMS/WCS service records). Field-level failures should reference the specific ISO element or FGDC path in their error messages to accelerate remediation. The metadata schema validation and linting workflow covers this in depth.

When datasets reference external services — WMS GetCapabilities URLs, linked open data endpoints, DOI-resolved landing pages — those references should be probed for HTTP 200 responses during the metadata stage. Storing these results in metadata artifact retention strategies records creates a long-term availability audit trail satisfying open-data portal SLAs.

# Multi-Jurisdictional and Interoperability Considerations

Geospatial data crosses political boundaries by nature, and the governance frameworks applied to it vary significantly across jurisdictions.

GDPR and location data. In EU jurisdictions, location data that can identify individuals — GPS tracks, precise address points, mobility datasets — is personal data under GDPR Article 4(1). CI pipelines that ingest such datasets must verify that privacy impact assessments are present in the metadata, that retention periods are declared, and that any cross-border transfer annotations reference the applicable adequacy decision or Standard Contractual Clauses. Automated checks should flag datasets containing point geometries at residential address precision for mandatory human review.

INSPIRE Directive. EU member states publishing spatial datasets under the 34 INSPIRE themes must comply with specific metadata profiles, service protocols (WMS, WFS, CSW), and data harmonization schemas. CI validation for INSPIRE-bound data adds a conformance check layer: metadata must satisfy both ISO 19115 and the INSPIRE Metadata Technical Guidance, which are related but not identical schemas.

NSDI alignment. The US National Spatial Data Infrastructure mandates that federal agencies publish discoverable, accessible, and interoperable geospatial data. CI pipelines for federal agencies should include a NSDI conformance check: clearinghouse-compatible metadata, published ISO 19115 records, and compliance with OMB Circular A-16 data asset inventory requirements.

Open data directives. Many state and municipal governments now mandate open data publication, often requiring CC-BY or CC0 licenses and DCAT-compatible catalog entries. Pipelines serving these contexts must enforce license declarations and catalog record completeness as hard gates rather than advisory warnings. The creative commons licensing for GIS datasets guide covers permissible license chains and attribution requirements.

Cross-format interoperability. When the same dataset must be published in multiple formats — GeoPackage for desktop GIS, GeoJSON for web applications, Cloud-Optimized GeoTIFF for raster analytics — the CI pipeline should validate each derivative independently. CRS handling, attribute type fidelity, and geometry precision can diverge across format conversions. Including format-conversion validation in the pipeline catches regressions before derivatives reach downstream consumers.

# Compliance Checklist

Use this checklist to audit your current spatial data pipeline against the governance requirements covered in this guide.

Version control and data storage

• Spatial files are stored in version-controlled repositories (Git LFS, DVC, or equivalent)
• Data registries enforce immutable versioning — once a dataset version is published, it cannot be overwritten in place
• Large file checkouts are optimized with pointer files; CI runners do not clone entire dataset history on every run

Schema and structural validation

• A YAML or JSON manifest declares required columns, accepted CRS codes, geometry types, and file size limits for each dataset family
• Schema linting runs on every pull request and fails the pipeline on any structural violation
• CRS is declared explicitly using an EPSG code or full WKT in every dataset; implicit CRS assumptions are prohibited
• Geometry validity is checked against OGC Simple Features on every changed file

Metadata compliance

• ISO 19115 or FGDC metadata is present for every dataset and validated field-by-field in CI
• SPDX license identifiers are present in the data_license SPDX license identifiers are present in the `data_license` field and validated against an organizational allowlist
• External service references (WMS, WFS, linked data URIs) are probed for availability during validation
• Metadata artifacts from every CI run are stored in immutable storage with timestamps and commit hashes

Policy enforcement

• Policy rules are defined as code (YAML or Python) and version-controlled alongside the dataset repository
• Hard violations (missing license, invalid CRS, topology failure) block pull request merges automatically
• High-risk changes (large boundary shifts, sensitive demographic data) route to designated stewards via CODEOWNERS
• Policy exceptions are logged with the approver's identity, timestamp, and justification

Multi-jurisdictional and licensing

• Location data capable of identifying individuals is flagged for GDPR review before ingestion
• INSPIRE or NSDI conformance checks are applied to datasets destined for those registries
• Share-alike license contamination (ODbL, CC-BY-SA) is checked before combining datasets with differing license terms
• Attribution strings are generated and verified as part of the validation pipeline

Audit and observability

• Every CI run produces a signed, timestamped JSON report covering all stages
• Validation reports reference specific ISO 19115 elements or FGDC paths for each failure
• Pipeline duration, failure rates, and false-positive frequencies are monitored over time
• Immutable audit logs can be produced within 24 hours for FOIA requests or legal review

# Related

• Spatial Data Schema Linting in CI — implement CRS, column, and geometry-type constraints as automated CI checks
• Policy Enforcement Gates for Data PRs — configure hard blocks and steward-approval routing for spatial data pull requests
• Automated Broken Link and Reference Detection — probe metadata service URLs and cross-dataset foreign keys before publication
• Metadata Artifact Retention Strategies — store and version compliance reports for long-term audit access
• Automated Metadata Generation & Schema Mapping — the companion domain covering ISO 19115, FGDC, and DCAT-AP metadata pipeline automation
• Geospatial Data Licensing & Compliance Fundamentals — licensing frameworks, risk scoring, and attribution requirements for spatial datasets