# Building a License Compliance Matrix for Municipal Data

Build a license compliance matrix for municipal data by mapping each dataset’s URI, spatial extent, and license text into a structured, machine-readable registry that automatically computes boolean compliance flags, schedules audit dates, and blocks redistribution of restricted assets.

Municipal GIS environments are inherently mixed-licensing. Parcel records, zoning overlays, satellite imagery, and IoT sensor feeds arrive from different agencies, each carrying distinct obligations. Automated attribution mapping workflows exist precisely to convert this legal complexity into queryable pipeline metadata, and this page shows how to build the compliance matrix that feeds those workflows. For the broader regulatory and risk context, see Geospatial Data Licensing & Compliance Fundamentals.

The core challenge is normalization: a municipal open-data portal might describe the same license as Creative Commons Attribution 4.0 International, CC BY 4.0, or simply open for public use. Without a deterministic parser that maps these variants to a canonical SPDX identifier and attaches boolean flags, every downstream pipeline step must re-interpret legal text — a process that does not scale and produces inconsistent decisions.

# Compliance Matrix Schema

A functional matrix bridges legal obligations and operational metadata. The schema below encodes everything a pipeline needs to make autonomous compliance decisions:

The metadata_confidence score is what separates a working matrix from a fragile one. A full regex match on a clean SPDX string earns 0.95; a partial pattern match on an informal phrase earns 0.80; a failed classification produces 0.0. Any row below 0.75 must be queued for manual legal review before the dataset enters a production pipeline.

The diagram below shows how a raw license string flows through the parser into the compliance decision:

# Automated Python Implementation

The script below is a single, self-contained module. It ingests a CSV metadata export, normalizes license strings to SPDX identifiers with confidence scores, applies the compliance rule engine, and writes the resulting matrix to CSV. It handles missing values, mixed-case input, and multi-word license descriptions.

"""
license_compliance_matrix.py

Build a license compliance matrix from a raw municipal metadata CSV.
Required input columns: dataset_id, source_agency, raw_license, license_url
Output: license_compliance_matrix.csv
"""

import pandas as pd
import re
from datetime import datetime, timedelta
from typing import Tuple, Dict

# -------------------------------------------------------------------
# 1. SPDX normalization map
#    Keys are case-insensitive regex patterns; values are SPDX IDs.
#    Order matters: longer/more-specific patterns first.
# -------------------------------------------------------------------
LICENSE_PATTERNS: Dict[str, str] = {
    r'(?i)cc\s*by[\s-]sa\s*4\.0': 'CC-BY-SA-4.0',
    r'(?i)cc\s*by\s*4\.0|creative\s*commons\s*attribution\s*4': 'CC-BY-4.0',
    r'(?i)odbl[\s-]*1\.0|open\s*database\s*licen[cs]e': 'ODbL-1.0',
    r'(?i)cc\s*0|cc\s*zero|public\s*domain|unlicense': 'CC0-1.0',
    r'(?i)open\s*government\s*licen[cs]e': 'OGL-3.0',
    r'(?i)proprietary|all\s*rights\s*reserved|commercial\s*licen[cs]e': 'Proprietary',
}

# -------------------------------------------------------------------
# 2. Compliance rule engine
#    Maps each SPDX ID to boolean flags and pipeline status.
# -------------------------------------------------------------------
COMPLIANCE_RULES: Dict[str, Dict] = {
    'CC-BY-4.0': {
        'attribution_required': True,
        'commercial_use_allowed': True,
        'share_alike_trigger': False,
        'compliance_status': 'Compliant',
        'review_days': 365,
    },
    'CC-BY-SA-4.0': {
        'attribution_required': True,
        'commercial_use_allowed': True,
        'share_alike_trigger': True,
        'compliance_status': 'Review_Required',
        'review_days': 90,
    },
    'ODbL-1.0': {
        'attribution_required': True,
        'commercial_use_allowed': True,
        'share_alike_trigger': True,
        'compliance_status': 'Review_Required',
        'review_days': 90,
    },
    'CC0-1.0': {
        'attribution_required': False,
        'commercial_use_allowed': True,
        'share_alike_trigger': False,
        'compliance_status': 'Compliant',
        'review_days': 365,
    },
    'OGL-3.0': {
        'attribution_required': True,
        'commercial_use_allowed': True,
        'share_alike_trigger': False,
        'compliance_status': 'Compliant',
        'review_days': 365,
    },
    'Proprietary': {
        'attribution_required': False,
        'commercial_use_allowed': False,
        'share_alike_trigger': False,
        'compliance_status': 'Blocked',
        'review_days': 90,
    },
    'Unknown': {
        'attribution_required': False,
        'commercial_use_allowed': False,
        'share_alike_trigger': False,
        'compliance_status': 'Unknown',
        'review_days': 90,
    },
}


def classify_license(raw: str) -> Tuple[str, float]:
    """
    Normalize a raw license string to an SPDX ID with a confidence score.

    Confidence levels:
      0.95  full regex match on canonical SPDX string
      0.80  partial regex match (substring found)
      0.0   no pattern matched → Unknown
    """
    if pd.isna(raw) or str(raw).strip() == '':
        return 'Unknown', 0.0

    text = str(raw).strip()
    for pattern, spdx_id in LICENSE_PATTERNS.items():
        if re.fullmatch(pattern, text):
            return spdx_id, 0.95
        if re.search(pattern, text):
            return spdx_id, 0.80

    return 'Unknown', 0.0


def compute_review_date(days_out: int) -> str:
    return (datetime.now() + timedelta(days=days_out)).strftime('%Y-%m-%d')


def build_compliance_matrix(df: pd.DataFrame) -> pd.DataFrame:
    """
    Apply normalization and the compliance rule engine to a metadata DataFrame.

    Returns a new DataFrame with all compliance columns appended.
    Rows with metadata_confidence < 0.75 are flagged for manual review
    via compliance_status = 'Unknown' regardless of rule-engine output.
    """
    # Step A: Normalize license strings
    classified = df['raw_license'].apply(classify_license)
    df['license_type'] = classified.apply(lambda t: t[0])
    df['metadata_confidence'] = classified.apply(lambda t: t[1])

    # Step B: Apply rule engine row by row
    rule_cols = [
        'attribution_required', 'commercial_use_allowed',
        'share_alike_trigger', 'compliance_status', 'review_days',
    ]
    rules_df = pd.DataFrame.from_dict(COMPLIANCE_RULES, orient='index')[rule_cols]
    df = df.join(rules_df, on='license_type')

    # Step C: Override low-confidence rows to Unknown / Review_Required
    low_conf = df['metadata_confidence'] < 0.75
    df.loc[low_conf, 'compliance_status'] = 'Unknown'

    # Step D: Compute review dates from per-row review_days
    df['review_date'] = df['review_days'].apply(
        lambda d: compute_review_date(int(d)) if pd.notna(d) else compute_review_date(90)
    )

    output_cols = [
        'dataset_id', 'source_agency', 'license_type', 'license_url',
        'attribution_required', 'commercial_use_allowed', 'share_alike_trigger',
        'review_date', 'compliance_status', 'metadata_confidence',
    ]
    return df[[c for c in output_cols if c in df.columns]]


if __name__ == '__main__':
    import sys

    input_csv = sys.argv[1] if len(sys.argv) > 1 else 'municipal_metadata_export.csv'
    output_csv = sys.argv[2] if len(sys.argv) > 2 else 'license_compliance_matrix.csv'

    raw_data = pd.read_csv(input_csv)
    matrix = build_compliance_matrix(raw_data)
    matrix.to_csv(output_csv, index=False)

    # Print a summary to stdout for CI logs
    status_counts = matrix['compliance_status'].value_counts().to_dict()
    print(f"Matrix written to {output_csv}")
    print(f"Status summary: {status_counts}")

    blocked = (matrix['compliance_status'] == 'Blocked').sum()
    unknown = (matrix['compliance_status'] == 'Unknown').sum()
    if blocked > 0 or unknown > 0:
        print(f"WARNING: {blocked} Blocked and {unknown} Unknown rows require review.")
        sys.exit(1)

The sys.exit(1) on any Blocked or Unknown row integrates directly into a CI gate. The automated attribution mapping workflows that generate citation strings read from the attribution_required column in this output; only rows where that flag is True and compliance_status is Compliant or Review_Required proceed to citation generation.

# Spatial and temporal extensions

Municipal licenses frequently include spatial and temporal constraints that simple boolean flags cannot capture. Extend the schema with three additional columns when your portfolio includes datasets that carry these terms:

• spatial_bbox (GeoJSON geometry string) — the bounding box within which the license terms apply; data used outside this extent may require a separate agreement
• valid_until (ISO 8601 date or empty) — sunset clause date, after which the dataset reverts to Unknown status automatically
• derivative_threshold (integer or null) — maximum number of parcels or features that may be aggregated before copyleft obligations trigger

Add a post-processing step that sets compliance_status = 'Unknown' for any row where valid_until is non-null and has passed datetime.now().

# Validation & Pipeline Integration

After generating the matrix, verify it before integrating it into any downstream pipeline step.

Schema validation with pandas

import pandas as pd

matrix = pd.read_csv('license_compliance_matrix.csv')

# Required columns must be present
required_cols = {
    'dataset_id', 'source_agency', 'license_type',
    'attribution_required', 'commercial_use_allowed',
    'share_alike_trigger', 'review_date',
    'compliance_status', 'metadata_confidence',
}
missing = required_cols - set(matrix.columns)
assert not missing, f"Missing columns: {missing}"

# No nulls in critical decision columns
assert matrix['compliance_status'].notna().all(), "Null compliance_status found"
assert matrix['attribution_required'].notna().all(), "Null attribution_required found"

# Confidence scores in valid range
assert matrix['metadata_confidence'].between(0.0, 1.0).all(), \
    "metadata_confidence out of [0.0, 1.0] range"

# No Blocked rows in production pipeline
blocked = matrix[matrix['compliance_status'] == 'Blocked']
assert blocked.empty, f"Blocked datasets found:\n{blocked[['dataset_id','source_agency','license_type']]}"

print("Matrix validation passed.")

CI gate in GitHub Actions

steps:
  - name: Validate license compliance matrix
    run: |
      python license_compliance_matrix.py municipal_metadata_export.csv license_compliance_matrix.csv
      python -c "
      import pandas as pd, sys
      m = pd.read_csv('license_compliance_matrix.csv')
      blocked = (m['compliance_status'] == 'Blocked').sum()
      unknown = (m['compliance_status'] == 'Unknown').sum()
      if blocked or unknown:
          print(f'CI FAIL: {blocked} Blocked, {unknown} Unknown rows')
          sys.exit(1)
      print('Compliance gate passed.')
      "

For spatial data schema validation alongside license checks, pair this gate with spatial data schema linting in CI to catch both metadata compliance failures and geometry format violations in a single pipeline run.

Review alert scheduling

Configure a nightly cron job (or a GitHub Actions scheduled workflow) to re-run the matrix generator against the live metadata export. Rows whose review_date has passed automatically surface in the next run with degraded confidence, preventing silently expired compliance records from propagating into production data products.

# Long-term Compliance Best Practices

• Version the matrix alongside the data. Store license_compliance_matrix.csv in the same repository as the datasets it governs. Use a git log or database changelog to surface when a license classification changed, satisfying audit requests without manual reconstruction.
• Pin the SPDX lookup table to a known release. The LICENSE_PATTERNS dictionary should reference a dated version of the SPDX license list. When the list updates, diff the new identifiers against your existing license_type column before deploying the updated parser.
• Treat Unknown as Blocked for publication gates. A dataset classified as Unknown carries undefined redistribution obligations. Default to the most restrictive posture in automated decisions; require explicit sign-off from a data steward before the dataset is allowed through a publication gate.
• Extend the confidence threshold as your corpus grows. The 0.75 threshold is a starting point for new implementations. As you accumulate a labelled ground-truth set of license strings from your specific agency portfolio, tune the threshold against precision/recall on that set.
• Record lineage when datasets are merged. A spatial join between a CC-BY-4.0 layer and an ODbL-1.0 layer produces a derivative whose most restrictive source governs the output. Implement a lineage_ids column that stores a comma-separated list of source dataset_id values; the matrix generator can then compute the combined share_alike_trigger as a logical OR across all parents. This aligns with the commercial EULA compliance tracking patterns used for vendor-licensed layers.
• Schedule Proprietary rows for contract renewal review. Vendor data products under proprietary terms often expire or change scope at contract renewal. Flag compliance_status = 'Blocked' entries for calendar review 60 days before the estimated contract end date to prevent operational disruption.

# Related

• Automated Attribution Mapping Workflows — parent cluster covering the full pipeline from metadata ingestion to citation string generation
• Geospatial Data Licensing & Compliance Fundamentals — the licensing domain overview covering SPDX, open-government licenses, and risk surface
• Commercial EULA Compliance Tracking — extends the matrix approach to vendor and proprietary data layers
• Spatial Data Schema Linting in CI — how to gate pipelines on both license compliance and geometry/schema validity in a single workflow