# GitHub Actions Workflows for Spatial Data

A workflow that validates spatial data behaves differently from one that tests an application: the inputs are large, the toolchain is heavy, and most runs should do almost nothing because most commits touch no data at all. This topic sits under CI/CD Validation & Policy Enforcement for Spatial Data and covers the workflow structure, dependency handling and reporting that make spatial checks fast enough to keep.

# Prerequisites

1. A repository where spatial data or metadata is version controlled, whether as files or as migration scripts.
2. A pinned Python environment — a lock file, not a range — since GDAL and its bindings are the least forgiving dependency pair in the ecosystem.
3. Checks that can run on a subset of files. A validator that only operates on a whole dataset cannot be wired to a diff, which is the single biggest determinant of workflow cost.
4. A decision about where results are reported, covered in posting validation results as pull request comments.

# Concept & Spec Reference

Four workflow features do most of the work in a spatial repository, and each has a failure mode worth knowing before it is adopted.

The first row deserves elaboration because it produces the most confusing outcome. A branch protection rule requiring a check that is skipped by a path filter will wait for a status that never arrives, and the pull request cannot be merged. The resolution is a companion job with the same name that runs on the inverse path filter and reports success immediately — an idiom that looks redundant and is load-bearing.

# Implementation Walkthrough

# Step 1 — Filter before doing anything

The cheapest run is the one that does not happen. Filtering at the workflow level rather than inside the job means no runner is provisioned at all.

WORKFLOW = """
name: spatial-validation

on:
  pull_request:
    paths:
      - 'data/**'
      - 'metadata/**'
      - '.github/workflows/spatial-validation.yml'

concurrency:
  group: spatial-validation-${{ github.ref }}
  cancel-in-progress: true
"""

The concurrency block matters more on data repositories than on code ones, because a push that supersedes an in-flight run would otherwise leave two jobs reading the same large files.

# Step 2 — Resolve the changed files once

Every subsequent step consumes the same list, so compute it once and pass it along rather than each step running its own diff.

import subprocess

SPATIAL_SUFFIXES = {".gpkg", ".geojson", ".shp", ".tif", ".xml", ".json"}


def changed_spatial_files(base_ref, head_ref="HEAD"):
    """Paths changed between two refs, filtered to spatial and metadata files."""
    out = subprocess.run(
        ["git", "diff", "--name-only", "--diff-filter=ACMR", f"{base_ref}...{head_ref}"],
        capture_output=True, text=True, check=True,
    ).stdout.splitlines()
    return [p for p in out if any(p.endswith(s) for s in SPATIAL_SUFFIXES)]

The --diff-filter=ACMR excludes deletions, which have nothing to validate — a deleted file that a validator tries to open produces a confusing failure that looks like corruption.

# Step 3 — Run the checks against that list

def run_checks(paths, checks):
    """Every check sees every path; results are collected, not raised."""
    results = []
    for path in paths:
        for check in checks:
            try:
                findings = list(check(path))
            except Exception as exc:  # a broken check is a check failure, not a data failure
                results.append({"path": path, "check": check.__name__,
                                "status": "error", "detail": str(exc)})
                continue
            results.append({
                "path": path,
                "check": check.__name__,
                "status": "fail" if findings else "pass",
                "findings": [f.__dict__ if hasattr(f, "__dict__") else f for f in findings],
            })
    return results


def summarise(results):
    return {
        "checked": len({r["path"] for r in results}),
        "failed": sum(1 for r in results if r["status"] == "fail"),
        "errored": sum(1 for r in results if r["status"] == "error"),
    }

Distinguishing fail from error is the detail that keeps the workflow honest. A check that crashed has told you nothing about the data, and reporting it as a data failure sends the wrong team to investigate.

# Step 4 — Report and retain

def to_annotations(results):
    """GitHub workflow commands: one annotation per finding, on the file."""
    lines = []
    for result in results:
        if result["status"] != "fail":
            continue
        for finding in result.get("findings", []):
            message = finding.get("message", "validation failed")
            lines.append(f"::error file={result['path']}::{message}")
    return "\n".join(lines)

# Validation & CI Integration

The workflow itself needs testing, and the parts worth testing are the ones that are easy to get subtly wrong.

def test_deletions_are_excluded():
    paths = changed_spatial_files("HEAD~1")
    assert all(not p.startswith("deleted/") for p in paths)


def test_broken_check_reports_error_not_fail():
    def broken(_path):
        raise RuntimeError("driver missing")
    results = run_checks(["a.gpkg"], [broken])
    assert results[0]["status"] == "error"


def test_summary_counts_paths_not_results():
    results = [{"path": "a.gpkg", "check": "x", "status": "pass"},
               {"path": "a.gpkg", "check": "y", "status": "fail"}]
    assert summarise(results)["checked"] == 1

Run the workflow against a fixture branch containing a deliberately invalid file as part of the repository’s own tests. A validation workflow that has never failed on anything is not evidence that the data is clean; it is evidence of nothing at all, and the only way to distinguish the two is to feed it something it must reject.

# Derivative & Lineage Management

A workflow run is evidence, and treating it that way costs almost nothing at the time and is impossible to reconstruct later.

Three facts make a run citable. The commit SHA the checks ran against, which the workflow already knows. The versions of the checking tools, which a lock file supplies and which decide whether a verdict from six months ago means the same thing today. And the check set that ran, because a run in which two checks were skipped for lack of a dependency is not the same run as one where all four executed, and nothing in a green tick distinguishes them.

Emitting those three into the retained report turns the artifact into something audit trail and evidence retention can consume. Without them the artifact records that something passed, which is the least useful true statement available.

Retention should follow the schedule described in metadata artifact retention strategies: the verdicts kept long, the payloads kept briefly.

# Runner Cost and Where It Actually Goes

Teams routinely optimise the wrong part of a spatial workflow, because the intuition that “validating the data must be the slow part” is almost never true. On a typical run the check itself is seconds and everything around it is minutes.

Environment construction dominates. Installing GDAL and its Python bindings from source takes minutes; installing from a wheel takes tens of seconds; restoring a warm cache takes a few. The spread between the worst and best case here is larger than the entire rest of the workflow, which is why caching GDAL and Python dependencies is the first optimisation worth making and usually the last one needed.

Checkout is the second cost, and it is avoidable. A repository holding data files has a large history, and a full clone fetches all of it to read four changed files. fetch-depth set to the minimum the diff requires — typically the merge base plus one — turns a two-minute checkout into a five-second one. Where the data itself is large, keeping it out of git history entirely and referencing it by URL changes the profile completely.

Schema and vocabulary fetches are slow and flaky. A validation run that resolves an XSD set or an RDF vocabulary over the network pays tens of seconds for it, fails intermittently when the remote is slow, and fails completely on runners without egress. Vendoring those files is both faster and the difference between a workflow that works offline and one that does not.

The check itself is proportional to the diff. Once the environment is warm, validating four changed files takes about as long as opening them. This is the part teams try to parallelise, and it is the part where parallelism buys the least — a matrix over four small files spends more time provisioning runners than it saves.

The practical consequence is an ordering. Fix the cache, then the checkout depth, then vendor the external schemas, and only then consider whether the checks themselves need to be faster. Most spatial workflows never reach the fourth step.

# Deciding What Runs Where

Not every check belongs in a pull request workflow, and the distinction is about what the check needs rather than how long it takes.

Checks that read a single file and need no external state belong on the pull request: schema conformance, geometry validity, required metadata fields. They are fast, they are attributable to the change, and a failure is the author’s to fix.

Checks that need the whole dataset — coverage gaps, cross-layer topology, catalogue completeness — belong on a scheduled run over the default branch. Running them per pull request means every author is shown failures caused by other people’s data, which is the fastest way to make a check ignored.

Checks that depend on the outside world — resolving distribution URLs, re-resolving upstream licences — belong on a schedule regardless of speed, because a pull request that fails when a remote host is down is a pull request that fails for reasons the author cannot act on.

# Permissions and What a Workflow Should Be Allowed to Do

A spatial validation workflow reads files and writes comments. It does not need write access to the repository, package registries or deployment credentials, and granting them by default is how a data repository acquires a supply chain problem.

Start from a read-only default and add what each job needs. A top-level permissions: contents: read with per-job additions makes the grants visible in review. A workflow that posts pull request comments needs pull-requests: write and nothing else; one that publishes an artifact needs no additional grant at all.

Be careful with pull_request_target. It exists because pull_request runs from a fork have no secrets and cannot comment, and it solves that by running the base branch’s workflow with full permissions in the context of the fork’s changes. Used carelessly — checking out the fork’s code and executing it — it hands write access to anyone who opens a pull request. For spatial validation the safe pattern is to keep the checking job on pull_request with no secrets, upload its report as an artifact, and have a separate workflow_run job post the results.

Pin third-party actions to a commit SHA, not a tag. A tag is mutable, and an action that reads your data files and has network access is well placed to exfiltrate them. This matters more in a data repository than in a code one, because the interesting content is the thing being checked.

Treat the data as untrusted input. A GeoPackage from a fork is a SQLite file that a validator will open, and a metadata document is XML that a parser will read. Entity resolution disabled, a size limit before parsing, and no shelling out with paths interpolated into a command string cover most of what can go wrong.

None of this is specific to spatial work, but the consequences are: the repository contains the data, and a workflow that can be induced to leak or corrupt it is more costly than one that can only break a build.

# Pitfalls & Resolution Table

# Related

• Caching GDAL and Python Dependencies in GitHub Actions — the largest single cost in a spatial workflow
• Running Spatial Validation on a Matrix of Formats — one check set, several drivers
• Posting Validation Results as Pull Request Comments — reporting where the author will read it
• Policy Enforcement Gates for Data PRs — the policy layer these workflows execute
• CI/CD Validation & Policy Enforcement for Spatial Data — the parent guide to gates, stages and enforcement