Geospatial Compliance Metadata Automation Hub

Detecting Invalid Geometries with Shapely in CI

To detect invalid geometries in CI, read each changed vector file into a geopandas GeoDataFrame and test every feature with shapely.is_valid, reporting the reason with explain_validity and exiting with a non-zero status so the continuous integration job fails the pull request. That single pattern — scan, classify, exit — is all a merge gate needs to stop self-intersecting polygons and other malformed shapes from reaching a production catalog.

This guide is the focused, copy-ready implementation of one check within Automated Topology & Geometry Validation, the parent guide that also covers coverage topology (gaps and overlaps) and repair previews. It sits inside the broader CI/CD Validation & Policy Enforcement for Spatial Data framework, which decides when a failing geometry check blocks a merge versus merely warns. Here the scope is deliberately narrow: object-level validity of individual geometries, done well.

Automated Python Implementation

The script below is self-contained. Point it at one or more vector files (GeoPackage, GeoJSON, Shapefile, FlatGeobuf) and it prints a report of every invalid feature and returns exit code 1 if any are found. It relies only on geopandas and shapely — both ship manylinux wheels with GEOS bundled, so no system GDAL install is required on the runner.

#!/usr/bin/env python3
"""Detect invalid geometries in vector datasets for CI gating.

Usage:
    python detect_invalid_geometries.py data/parcels.gpkg data/zoning.geojson

Exit codes:
    0 — every geometry in every file is valid and non-empty
    1 — at least one invalid, empty, or null geometry was found
    2 — a file could not be read
"""
import sys
from pathlib import Path

import geopandas as gpd
import shapely
from shapely.validation import explain_validity


def scan_file(path: Path) -> list[dict]:
    """Return a list of violation records for one vector file."""
    gdf = gpd.read_file(path, engine="pyogrio")
    violations: list[dict] = []

    for fid, geom in zip(gdf.index, gdf.geometry):
        if geom is None:
            violations.append({"fid": int(fid), "reason": "null geometry", "at": None})
            continue
        if shapely.is_empty(geom):
            violations.append({"fid": int(fid), "reason": "empty geometry", "at": None})
            continue
        if not shapely.is_valid(geom):
            reason = explain_validity(geom)
            # explain_validity returns e.g. "Self-intersection[537402.1 182004.7]"
            at = None
            if "[" in reason and reason.endswith("]"):
                at = reason[reason.index("[") + 1: -1]
            violations.append({"fid": int(fid), "reason": reason, "at": at})

    return violations


def report(path: Path, violations: list[dict]) -> None:
    """Print a human-readable report for one file."""
    if not violations:
        print(f"OK    {path} — all geometries valid")
        return
    print(f"FAIL  {path} — {len(violations)} invalid geometr(ies):")
    for v in violations:
        location = f" at {v['at']}" if v["at"] else ""
        print(f"        feature {v['fid']}: {v['reason']}{location}")


def main(argv: list[str]) -> int:
    paths = [Path(p) for p in argv[1:]]
    if not paths:
        print("usage: detect_invalid_geometries.py FILE [FILE ...]")
        return 2

    total_invalid = 0
    for path in paths:
        if not path.exists():
            print(f"ERROR file not found: {path}")
            return 2
        try:
            violations = scan_file(path)
        except Exception as exc:  # noqa: BLE001 — surface any read failure as exit 2
            print(f"ERROR could not read {path}: {exc}")
            return 2
        report(path, violations)
        total_invalid += len(violations)

    print(f"\nScanned {len(paths)} file(s); {total_invalid} invalid geometr(ies) total.")
    return 1 if total_invalid else 0


if __name__ == "__main__":
    sys.exit(main(sys.argv))

Two design choices make this CI-friendly. First, it collects all violations in one pass instead of aborting on the first, so a contributor sees the complete list of what to fix. Second, it distinguishes three outcomes through exit codes — clean (0), invalid data (1), and operational error such as an unreadable file (2) — so a broken runner is never mistaken for clean data.

Validation and pipeline integration

Run the script locally against a file you know is clean and one you know is broken before trusting it in CI. A quick way to manufacture a broken fixture is a bowtie polygon written with shapely:

# Install the pinned toolchain
pip install shapely==2.0.4 geopandas==1.0.1 pyogrio==0.9.0

# Create a deliberately invalid fixture, then scan it
python - <<'PY'
import geopandas as gpd
import shapely
bowtie = shapely.from_wkt("POLYGON ((0 0, 1 1, 1 0, 0 1, 0 0))")
gpd.GeoDataFrame(geometry=[bowtie], crs="EPSG:27700").to_file("bad.gpkg")
PY

python detect_invalid_geometries.py bad.gpkg
echo "exit code: $?"   # expect 1

Lock the behaviour into a pytest case so a refactor cannot loosen it:

import geopandas as gpd
import shapely
from detect_invalid_geometries import scan_file


def test_bowtie_detected(tmp_path):
    bowtie = shapely.from_wkt("POLYGON ((0 0, 1 1, 1 0, 0 1, 0 0))")
    path = tmp_path / "bad.gpkg"
    gpd.GeoDataFrame(geometry=[bowtie], crs="EPSG:27700").to_file(path)
    violations = scan_file(path)
    assert len(violations) == 1
    assert "Self-intersection" in violations[0]["reason"]

Then trigger the scan from GitHub Actions on any change to a spatial file. The paths filter keeps the job off unrelated commits, and a non-zero exit fails the required check:

name: Detect Invalid Geometries
on:
  pull_request:
    paths:
      - "data/**/*.gpkg"
      - "data/**/*.geojson"
      - "data/**/*.shp"

jobs:
  geometry-validity:
    runs-on: ubuntu-24.04
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
      - uses: actions/setup-python@v5
        with:
          python-version: "3.11"
          cache: "pip"
      - name: Install
        run: pip install shapely==2.0.4 geopandas==1.0.1 pyogrio==0.9.0
      - name: Scan changed geometry files
        run: |
          git diff --name-only --diff-filter=ACMRT origin/${{ github.base_ref }} HEAD \
            | grep -E '\.(gpkg|geojson|shp)$' \
            | xargs -r python scripts/detect_invalid_geometries.py

Promote the geometry-validity job to a required status check under branch protection so it becomes an enforced merge gate rather than an advisory log line, exactly as described for the policy enforcement gates for data PRs.

Long-term compliance best practices

  • Pin shapely and assert its GEOS version. Validity semantics live in GEOS, not in Python. Add assert shapely.geos_version >= (3, 11, 0) at job startup so a runner with a different GEOS build fails loudly instead of producing results that disagree with a contributor’s laptop.
  • Report the coordinate, not just the reason. explain_validity returns the location of the defect in the layer’s own CRS. Surfacing it — as the script does — lets a reviewer jump straight to the offending vertex in QGIS instead of hunting through a large layer.
  • Never auto-repair inside the gate. make_valid changes area and can split features. Keep this check read-only; route repairs to a separate, reviewed commit and re-validate the result, as covered in the parent topology guide.
  • Treat empty geometries as failures. GEOS calls an empty geometry valid. For a production layer that is almost always a defect, so test shapely.is_empty explicitly rather than relying on is_valid alone.
  • Iterate multi-part features. explain_validity reports one reason for a MultiPolygon; loop over geom.geoms when you need a defect list for every part of a multi-part feature.
  • Re-validate after every transform. Reprojection, clipping, and simplification can each introduce invalidity into previously clean data, so run this scan again on any derivative before it is written, not only at initial ingest.