Geospatial Compliance Metadata Automation Hub

Mapping Commercial GIS Data Usage Rights: A Deterministic Pipeline

Mapping commercial GIS data usage rights means extracting license constraints from heterogeneous metadata formats, normalizing them into a machine-readable compliance matrix, and flagging violations automatically at dataset ingestion time — not during a post-hoc audit.

Commercial geospatial datasets rarely ship with standardized, machine-readable licenses. Rights are typically buried in PDF EULAs, proprietary catalog schemas, or fragmented XML blocks — which makes them invisible to automated pipelines unless you impose structure. For teams building commercial EULA compliance tracking workflows, manual EULA review introduces latency, audit exposure, and inconsistent enforcement. This page shows you how to build a deterministic pipeline that treats usage rights as structured, queryable data within the broader Geospatial Data Licensing & Compliance Fundamentals framework.

Pipeline Architecture

The three stages below form a sequential, deterministic flow. Each stage has a single, testable output.

The layers of a deterministic rights-mapping pipelineFour layers — intake, normalisation, classification and output — each with the artifacts it produces.Outputconsumed by CIrights recordaction flagevidence pathClassificationrights taxonomyordered patternsconfidenceNormalisationcase foldingwhitespacesynonymsIntakenever mutatedEULA textvendor metadatacontract id
Determinism comes from the layers never reaching past each other.

Stage 1 — Ingest & Parse: Pull raw license text from ISO 19139 XML useLimitation fields, GeoJSON properties, or vendor-specific JSON manifests. Use lxml for namespace-aware XML traversal; fall back gracefully when fields are absent.

Stage 2 — Normalize: Map extracted strings to a controlled vocabulary using compiled regex aliases. Align matches to SPDX identifiers (e.g., CC-BY-NC-4.0) or internal policy codes (e.g., Commercial-Restricted, Internal-Use-Only). Unmatched strings default to proprietary.

Stage 3 — Validate & Flag: Cross-reference normalized keys against an organizational rights matrix. Emit boolean compliance flags (allowed, requires_attribution, allows_derivatives). Datasets failing policy gates get blocked or routed to a legal review queue.

Automated Python Implementation

The script below is self-contained and runnable. It handles ISO 19139 XML and GeoJSON inputs, applies regex normalization, and returns a compliance record. It uses lxml for robust namespace-aware XML parsing and Python’s standard json and re modules — no additional dependencies beyond lxml.

The commercial rights taxonomy the classifier assignsFour rights classes compared by whether internal analysis, customer-facing publication and redistribution are permitted.Internal analysisCustomer-facing productRedistributionopen-datapermittedpermittedpermitted with noticelicensed-internalpermittedprohibitedprohibitedlicensed-externalpermittedpermitted, seat-cappedprohibitedunclassifiedholdholdhold
Four classes, three questions each — enough to route every dataset in a commercial catalogue.
import json
import re
from pathlib import Path
from lxml import etree
from typing import Dict, Optional

# SPDX-compatible commercial rights taxonomy.
# Add rows here as your catalog expands — never mutate existing keys.
RIGHTS_MATRIX: Dict[str, Dict[str, bool]] = {
    "cc-by-nc-4.0":   {"allowed": False, "requires_attribution": True,  "allows_derivatives": True},
    "commercial":     {"allowed": False, "requires_attribution": True,  "allows_derivatives": False},
    "internal-only":  {"allowed": False, "requires_attribution": False, "allows_derivatives": False},
    "open-data":      {"allowed": True,  "requires_attribution": True,  "allows_derivatives": True},
    "proprietary":    {"allowed": False, "requires_attribution": True,  "allows_derivatives": False},
}

# Order matters: more specific patterns must appear before broader ones.
LICENSE_ALIASES = [
    (r"(?i)cc.*by.*nc.*4\.0",               "cc-by-nc-4.0"),
    (r"(?i)internal.*use.*only",             "internal-only"),
    (r"(?i)commercial.*use.*only",           "commercial"),
    (r"(?i)open.*data|public.*domain",       "open-data"),
    (r"(?i)proprietary|all.*rights.*reserved", "proprietary"),
]
_COMPILED = [(re.compile(pat), key) for pat, key in LICENSE_ALIASES]

def normalize_license(raw_text: str) -> Optional[str]:
    """Map free-text license strings to a RIGHTS_MATRIX key.

    Returns None when raw_text is empty; callers should default to 'proprietary'.
    """
    if not raw_text:
        return None
    for pattern, key in _COMPILED:
        if pattern.search(raw_text):
            return key
    return None

def parse_iso19139(xml_path: Path) -> Optional[str]:
    """Extract useLimitation text from ISO 19139 XML.

    Tries namespace-qualified XPath first; falls back to local-name scan
    to handle broken or non-standard namespace declarations.
    """
    try:
        tree = etree.parse(str(xml_path))
    except etree.XMLSyntaxError:
        return None

    ns = {
        "gmd": "http://www.isotc211.org/2005/gmd",
        "gco": "http://www.isotc211.org/2005/gco",
    }
    xpath_candidates = [
        "//gmd:useLimitation/gco:CharacterString",
        "//*[local-name()='useLimitation']/*[local-name()='CharacterString']",
        "//*[local-name()='license']",
        "//*[local-name()='rightsStatement']",
    ]
    for xpath in xpath_candidates:
        nodes = tree.xpath(xpath, namespaces=ns)
        if nodes and nodes[0].text:
            return nodes[0].text.strip()
    return None

def parse_geojson(json_path: Path) -> Optional[str]:
    """Extract license/rights text from top-level GeoJSON properties.

    Scans a prioritized list of common rights-related keys; returns the
    first non-empty string found.
    """
    try:
        with open(json_path, "r", encoding="utf-8") as fh:
            data = json.load(fh)
    except (json.JSONDecodeError, OSError):
        return None

    props = data.get("properties", {})
    for key in ("license", "rights", "usage_rights", "eula",
                "terms_of_use", "vendor_license_url"):
        value = props.get(key)
        if isinstance(value, str) and value.strip():
            return value.strip()
    return None

def map_rights(metadata_path: Path) -> Dict:
    """Ingest metadata, normalize the license, and return a compliance record.

    The returned dict is safe to serialize directly to JSON and store
    alongside provenance records in your data catalog.
    """
    suffix = metadata_path.suffix.lower()

    if suffix in (".xml", ".iso19139"):
        raw_license = parse_iso19139(metadata_path)
    elif suffix == ".geojson":
        raw_license = parse_geojson(metadata_path)
    else:
        raw_license = None

    normalized_key = normalize_license(raw_license or "") or "proprietary"
    compliance = RIGHTS_MATRIX[normalized_key]

    return {
        "source_file":      str(metadata_path),
        "raw_license_text": raw_license or "NOT_FOUND",
        "normalized_id":    normalized_key,
        "compliance_flags": compliance,
        "action_required":  not compliance["allowed"],
    }

if __name__ == "__main__":
    import sys

    target = Path(sys.argv[1]) if len(sys.argv) > 1 else Path("dataset_metadata.xml")
    result = map_rights(target)
    print(json.dumps(result, indent=2))

The normalized_id field is the stable key to store in your catalog. raw_license_text is the verbatim string that produced it — keep both so a legal reviewer can verify or override the normalization without re-parsing the source file.

Validation & Pipeline Integration

After running the script, verify the output before propagating it downstream.

How a rights record reaches the CI gateThe extractor produces a rights record, the schema validator accepts or rejects it, and the CI gate reads the action flag.ExtractorSchema checkCI gaterights record + evidence pathvalidate against JSON Schemareject with field pathaccepted recordfail build if action_required
The gate never parses an EULA; it reads a validated record.

1. Smoke-test against known fixtures

# Install lxml once
pip install "lxml>=4.9"

# Run against a known-open fixture; expect normalized_id == "open-data"
python map_rights.py fixtures/open_dataset.geojson
# Expected: "action_required": false

# Run against a known-proprietary fixture
python map_rights.py fixtures/vendor_delivery.xml
# Expected: "action_required": true

2. JSON Schema validation

Validate every compliance record against a schema before writing it to your catalog store:

# pip install jsonschema
import jsonschema

RECORD_SCHEMA = {
    "type": "object",
    "required": ["source_file", "raw_license_text", "normalized_id",
                 "compliance_flags", "action_required"],
    "properties": {
        "normalized_id":    {"type": "string", "enum": list(RIGHTS_MATRIX.keys())},
        "action_required":  {"type": "boolean"},
        "compliance_flags": {
            "type": "object",
            "required": ["allowed", "requires_attribution", "allows_derivatives"],
        },
    },
}

def validate_record(record: dict) -> None:
    jsonschema.validate(instance=record, schema=RECORD_SCHEMA)

3. CI gate integration

In a GitHub Actions workflow, wire the script as a required check on any pull request that adds or modifies geospatial datasets. When action_required is true, exit with a non-zero code to block the merge. This pattern aligns directly with spatial data schema linting in CI practices for enforcement at the PR boundary.

# .github/workflows/rights-check.yml
- name: Map and validate usage rights
  run: |
    python map_rights.py "${{ env.DATASET_PATH }}" > rights_record.json
    python -c "
    import json, sys
    rec = json.load(open('rights_record.json'))
    if rec['action_required']:
        print(f'BLOCKED: {rec[\"normalized_id\"]} — route to legal review')
        sys.exit(1)
    print('OK:', rec['normalized_id'])
    "

Long-Term Compliance Best Practices

  • Pin your normalization dictionary to a versioned schema. SPDX identifiers evolve — document which SPDX release your LICENSE_ALIASES targets and store that version alongside your catalog configuration. Re-run normalization across existing records when you update the mapping.
  • Store both raw and normalized fields permanently. The raw_license_text value is your audit evidence. If a vendor later disputes your interpretation, the verbatim string is the defensible reference. Never discard it after normalization.
  • Default to the most restrictive classification on ambiguity. Any unmatched string should resolve to proprietary with action_required: true. Assuming permission where none is explicit is a compliance failure, not a safe fallback.
  • Extend the vendor registry incrementally. When a new commercial provider delivers data with non-standard keys, add their field names to parse_geojson() under a named registry entry. Version-control the registry so field-mapping changes are auditable.
  • Re-evaluate the matrix when policy changes, not just at ingestion. Store normalized records separately from raw metadata so a policy update (e.g., your organization now prohibits CC-BY-NC for model training) triggers a re-scan of existing catalog entries without re-parsing source files.
  • Coordinate rights mapping with automated attribution mapping workflows. The requires_attribution flag produced here is the upstream input for attribution-string generation — pass it downstream rather than re-deriving it from scratch.