Geospatial Compliance Metadata Automation Hub

Mapping Commercial GIS Data Usage Rights: A Deterministic Pipeline

Mapping commercial GIS data usage rights means extracting license constraints from heterogeneous metadata formats, normalizing them into a machine-readable compliance matrix, and flagging violations automatically at dataset ingestion time — not during a post-hoc audit.

Commercial geospatial datasets rarely ship with standardized, machine-readable licenses. Rights are typically buried in PDF EULAs, proprietary catalog schemas, or fragmented XML blocks — which makes them invisible to automated pipelines unless you impose structure. For teams building commercial EULA compliance tracking workflows, manual EULA review introduces latency, audit exposure, and inconsistent enforcement. This page shows you how to build a deterministic pipeline that treats usage rights as structured, queryable data within the broader Geospatial Data Licensing & Compliance Fundamentals framework.

Pipeline Architecture

The three stages below form a sequential, deterministic flow. Each stage has a single, testable output.

Commercial GIS rights-mapping pipeline Three sequential stages: (1) Ingest and Parse — ISO 19139 XML, GeoJSON, vendor JSON; (2) Normalize — regex alias matching to SPDX-compatible keys; (3) Validate and Flag — policy matrix cross-reference, compliance flags emitted. 1. Ingest & Parse ISO 19139 · GeoJSON vendor JSON 2. Normalize regex alias → SPDX key default: proprietary 3. Validate & Flag policy matrix lookup compliance flags emitted

Stage 1 — Ingest & Parse: Pull raw license text from ISO 19139 XML useLimitation fields, GeoJSON properties, or vendor-specific JSON manifests. Use lxml for namespace-aware XML traversal; fall back gracefully when fields are absent.

Stage 2 — Normalize: Map extracted strings to a controlled vocabulary using compiled regex aliases. Align matches to SPDX identifiers (e.g., CC-BY-NC-4.0) or internal policy codes (e.g., Commercial-Restricted, Internal-Use-Only). Unmatched strings default to proprietary.

Stage 3 — Validate & Flag: Cross-reference normalized keys against an organizational rights matrix. Emit boolean compliance flags (allowed, requires_attribution, allows_derivatives). Datasets failing policy gates get blocked or routed to a legal review queue.

Automated Python Implementation

The script below is self-contained and runnable. It handles ISO 19139 XML and GeoJSON inputs, applies regex normalization, and returns a compliance record. It uses lxml for robust namespace-aware XML parsing and Python’s standard json and re modules — no additional dependencies beyond lxml.

import json
import re
from pathlib import Path
from lxml import etree
from typing import Dict, Optional

# SPDX-compatible commercial rights taxonomy.
# Add rows here as your catalog expands — never mutate existing keys.
RIGHTS_MATRIX: Dict[str, Dict[str, bool]] = {
    "cc-by-nc-4.0":   {"allowed": False, "requires_attribution": True,  "allows_derivatives": True},
    "commercial":     {"allowed": False, "requires_attribution": True,  "allows_derivatives": False},
    "internal-only":  {"allowed": False, "requires_attribution": False, "allows_derivatives": False},
    "open-data":      {"allowed": True,  "requires_attribution": True,  "allows_derivatives": True},
    "proprietary":    {"allowed": False, "requires_attribution": True,  "allows_derivatives": False},
}

# Order matters: more specific patterns must appear before broader ones.
LICENSE_ALIASES = [
    (r"(?i)cc.*by.*nc.*4\.0",               "cc-by-nc-4.0"),
    (r"(?i)internal.*use.*only",             "internal-only"),
    (r"(?i)commercial.*use.*only",           "commercial"),
    (r"(?i)open.*data|public.*domain",       "open-data"),
    (r"(?i)proprietary|all.*rights.*reserved", "proprietary"),
]
_COMPILED = [(re.compile(pat), key) for pat, key in LICENSE_ALIASES]


def normalize_license(raw_text: str) -> Optional[str]:
    """Map free-text license strings to a RIGHTS_MATRIX key.

    Returns None when raw_text is empty; callers should default to 'proprietary'.
    """
    if not raw_text:
        return None
    for pattern, key in _COMPILED:
        if pattern.search(raw_text):
            return key
    return None


def parse_iso19139(xml_path: Path) -> Optional[str]:
    """Extract useLimitation text from ISO 19139 XML.

    Tries namespace-qualified XPath first; falls back to local-name scan
    to handle broken or non-standard namespace declarations.
    """
    try:
        tree = etree.parse(str(xml_path))
    except etree.XMLSyntaxError:
        return None

    ns = {
        "gmd": "http://www.isotc211.org/2005/gmd",
        "gco": "http://www.isotc211.org/2005/gco",
    }
    xpath_candidates = [
        "//gmd:useLimitation/gco:CharacterString",
        "//*[local-name()='useLimitation']/*[local-name()='CharacterString']",
        "//*[local-name()='license']",
        "//*[local-name()='rightsStatement']",
    ]
    for xpath in xpath_candidates:
        nodes = tree.xpath(xpath, namespaces=ns)
        if nodes and nodes[0].text:
            return nodes[0].text.strip()
    return None


def parse_geojson(json_path: Path) -> Optional[str]:
    """Extract license/rights text from top-level GeoJSON properties.

    Scans a prioritized list of common rights-related keys; returns the
    first non-empty string found.
    """
    try:
        with open(json_path, "r", encoding="utf-8") as fh:
            data = json.load(fh)
    except (json.JSONDecodeError, OSError):
        return None

    props = data.get("properties", {})
    for key in ("license", "rights", "usage_rights", "eula",
                "terms_of_use", "vendor_license_url"):
        value = props.get(key)
        if isinstance(value, str) and value.strip():
            return value.strip()
    return None


def map_rights(metadata_path: Path) -> Dict:
    """Ingest metadata, normalize the license, and return a compliance record.

    The returned dict is safe to serialize directly to JSON and store
    alongside provenance records in your data catalog.
    """
    suffix = metadata_path.suffix.lower()

    if suffix in (".xml", ".iso19139"):
        raw_license = parse_iso19139(metadata_path)
    elif suffix == ".geojson":
        raw_license = parse_geojson(metadata_path)
    else:
        raw_license = None

    normalized_key = normalize_license(raw_license or "") or "proprietary"
    compliance = RIGHTS_MATRIX[normalized_key]

    return {
        "source_file":      str(metadata_path),
        "raw_license_text": raw_license or "NOT_FOUND",
        "normalized_id":    normalized_key,
        "compliance_flags": compliance,
        "action_required":  not compliance["allowed"],
    }


if __name__ == "__main__":
    import sys

    target = Path(sys.argv[1]) if len(sys.argv) > 1 else Path("dataset_metadata.xml")
    result = map_rights(target)
    print(json.dumps(result, indent=2))

The normalized_id field is the stable key to store in your catalog. raw_license_text is the verbatim string that produced it — keep both so a legal reviewer can verify or override the normalization without re-parsing the source file.

Validation & Pipeline Integration

After running the script, verify the output before propagating it downstream.

1. Smoke-test against known fixtures

# Install lxml once
pip install "lxml>=4.9"

# Run against a known-open fixture; expect normalized_id == "open-data"
python map_rights.py fixtures/open_dataset.geojson
# Expected: "action_required": false

# Run against a known-proprietary fixture
python map_rights.py fixtures/vendor_delivery.xml
# Expected: "action_required": true

2. JSON Schema validation

Validate every compliance record against a schema before writing it to your catalog store:

# pip install jsonschema
import jsonschema

RECORD_SCHEMA = {
    "type": "object",
    "required": ["source_file", "raw_license_text", "normalized_id",
                 "compliance_flags", "action_required"],
    "properties": {
        "normalized_id":    {"type": "string", "enum": list(RIGHTS_MATRIX.keys())},
        "action_required":  {"type": "boolean"},
        "compliance_flags": {
            "type": "object",
            "required": ["allowed", "requires_attribution", "allows_derivatives"],
        },
    },
}

def validate_record(record: dict) -> None:
    jsonschema.validate(instance=record, schema=RECORD_SCHEMA)

3. CI gate integration

In a GitHub Actions workflow, wire the script as a required check on any pull request that adds or modifies geospatial datasets. When action_required is true, exit with a non-zero code to block the merge. This pattern aligns directly with spatial data schema linting in CI practices for enforcement at the PR boundary.

# .github/workflows/rights-check.yml
- name: Map and validate usage rights
  run: |
    python map_rights.py "${{ env.DATASET_PATH }}" > rights_record.json
    python -c "
    import json, sys
    rec = json.load(open('rights_record.json'))
    if rec['action_required']:
        print(f'BLOCKED: {rec[\"normalized_id\"]} — route to legal review')
        sys.exit(1)
    print('OK:', rec['normalized_id'])
    "

Long-Term Compliance Best Practices

  • Pin your normalization dictionary to a versioned schema. SPDX identifiers evolve — document which SPDX release your LICENSE_ALIASES targets and store that version alongside your catalog configuration. Re-run normalization across existing records when you update the mapping.
  • Store both raw and normalized fields permanently. The raw_license_text value is your audit evidence. If a vendor later disputes your interpretation, the verbatim string is the defensible reference. Never discard it after normalization.
  • Default to the most restrictive classification on ambiguity. Any unmatched string should resolve to proprietary with action_required: true. Assuming permission where none is explicit is a compliance failure, not a safe fallback.
  • Extend the vendor registry incrementally. When a new commercial provider delivers data with non-standard keys, add their field names to parse_geojson() under a named registry entry. Version-control the registry so field-mapping changes are auditable.
  • Re-evaluate the matrix when policy changes, not just at ingestion. Store normalized records separately from raw metadata so a policy update (e.g., your organization now prohibits CC-BY-NC for model training) triggers a re-scan of existing catalog entries without re-parsing source files.
  • Coordinate rights mapping with automated attribution mapping workflows. The requires_attribution flag produced here is the upstream input for attribution-string generation — pass it downstream rather than re-deriving it from scratch.