Geospatial Compliance Metadata Automation Hub

Detecting CC Licence Versions in Legacy Metadata

Scan every metadata carrier for a version token, resolve to a canonical SPDX identifier only when the version is explicit, and record anything that names a Creative Commons licence without a version as contested rather than guessing at the most likely one.

Inherited catalogues express Creative Commons licences in dozens of ways: a URL, a badge image filename, a sentence in an abstract, a .dbf column truncated to ten characters, a rights element containing the words “Creative Commons Attribution”. Some of these carry a version and some do not, and the difference matters more for spatial data than for most content, because version 4.0 extends its conditions to database rights while earlier versions do not. This guide sits under Creative Commons licensing for GIS datasets, within Geospatial Data Licensing & Compliance Fundamentals.

Where a Creative Commons claim hides in a legacy recordLicence claims appear in dedicated rights elements, in free-text abstracts, in file naming conventions and in attribute columns.A legacy metadata recordthe claim is somewhereRights elementsdesigned for thisuseConstraintsISO 19115dct:licenseDCAT, a URIFree textnot designed for thisAbstracta sentenceSupplemental infoa paragraphAttribute dataper featureLICENSE columntruncated at ten charsSOURCE columna URL, sometimesConventionsoutside the recordSidecar fileLICENSE.txtBadge imagea filename
Four hiding places, and only the first is designed to hold a licence.

Automated Python Implementation

The scanner below reads every carrier it can reach, extracts candidate claims, and classifies each as resolved, versionless or contested. It never picks a version that was not stated.

#!/usr/bin/env python3
"""Detect Creative Commons licence claims and their versions in legacy metadata."""
import pathlib
import re
from collections import Counter

# Canonical forms. A claim resolves only when the version is explicit.
FAMILIES = {
    "by-nc-sa": "CC-BY-NC-SA",
    "by-nc-nd": "CC-BY-NC-ND",
    "by-nc": "CC-BY-NC",
    "by-nd": "CC-BY-ND",
    "by-sa": "CC-BY-SA",
    "by": "CC-BY",
    "zero": "CC0",
    "publicdomain": "CC0",
}

URL_RE = re.compile(
    r"creativecommons\.org/(?:licenses|publicdomain)/([a-z\-]+)(?:/(\d\.\d))?"
    r"(?:/([a-z]{2}))?", re.I)
PROSE_RE = re.compile(
    r"creative\s+commons\s+((?:attribution|zero|public\s+domain)"
    r"(?:[\s\-]+(?:share[\s\-]?alike|non[\s\-]?commercial|no\s+deriv\w*))*)"
    r"(?:[^\d]{0,20}(\d\.\d))?", re.I)

PROSE_TOKENS = [
    ("non commercial", "nc"), ("noncommercial", "nc"),
    ("share alike", "sa"), ("sharealike", "sa"),
    ("no derivatives", "nd"), ("noderiv", "nd"),
    ("attribution", "by"), ("zero", "zero"), ("public domain", "publicdomain"),
]

def claims_from_text(text):
    """Yield (family, version, port, evidence) for every claim found."""
    for match in URL_RE.finditer(text):
        family = FAMILIES.get(match.group(1).lower())
        if family:
            yield family, match.group(2), match.group(3), match.group(0)

    for match in PROSE_RE.finditer(text):
        phrase = " ".join(match.group(1).lower().split())
        parts = [code for token, code in PROSE_TOKENS if token in phrase]
        if "zero" in parts or "publicdomain" in parts:
            family = "CC0"
        elif "by" in parts:
            family = "CC-BY" + "".join(
                f"-{p.upper()}" for p in ("nc", "sa", "nd") if p in parts)
        else:
            continue
        yield family, match.group(2), None, match.group(0)

def classify(claims):
    """Resolve to a single SPDX id, or report why not."""
    if not claims:
        return {"status": "none"}

    families = {c[0] for c in claims}
    if len(families) > 1:
        return {"status": "contested", "families": sorted(families),
                "evidence": [c[3] for c in claims]}

    family = families.pop()
    versions = {c[1] for c in claims if c[1]}
    ports = {c[2] for c in claims if c[2]}
    if len(versions) > 1:
        return {"status": "contested", "versions": sorted(versions),
                "evidence": [c[3] for c in claims]}
    if not versions:
        return {"status": "versionless", "family": family,
                "evidence": [c[3] for c in claims]}

    version = versions.pop()
    spdx = f"{family}-{version}"
    if ports:
        spdx += "-" + sorted(ports).pop().upper()
    return {"status": "resolved", "spdx": spdx, "evidence": [c[3] for c in claims]}

def scan(root):
    results, counts = {}, Counter()
    for path in sorted(pathlib.Path(root).rglob("*")):
        if path.suffix.lower() not in {".xml", ".txt", ".json", ".md"}:
            continue
        text = path.read_text(encoding="utf-8", errors="replace")
        outcome = classify(list(claims_from_text(text)))
        results[str(path)] = outcome
        counts[outcome["status"]] += 1
    return results, counts

The versionless status is the whole point of the exercise. A record reading “Creative Commons Attribution” is compatible with 1.0 through 4.0, and those differ on the question that decides whether the licence reaches a feature table at all. Resolving it to CC-BY-4.0 because that is the current version invents a fact; resolving it to CC-BY-3.0 because the record is old invents a different one. Reporting it as versionless is the only honest option, and it produces a work list rather than a false clean bill.

What each detected form actually tells youDetected claim forms compared by whether they carry a version, a jurisdiction port and enough information to resolve an SPDX identifier.Carries versionCarries portResolvesFull licence URLyessometimesyesURL without versionnonoversionlessProse with a numberusuallynoyes, if unambiguousProse without a numbernonoversionlessTruncated .dbf valuerarelynoversionless
Only the first row resolves on its own. The rest are evidence, not answers.

Validation and Pipeline Integration

The scanner needs a fixture corpus more than it needs unit tests, because the thing being tested is coverage of forms found in the wild.

def test_versioned_url_resolves():
    claims = list(claims_from_text("http://creativecommons.org/licenses/by-sa/4.0/"))
    assert classify(claims)["spdx"] == "CC-BY-SA-4.0"

def test_unversioned_prose_is_not_guessed():
    claims = list(claims_from_text("Released under a Creative Commons Attribution licence."))
    outcome = classify(claims)
    assert outcome["status"] == "versionless"
    assert "spdx" not in outcome

def test_disagreeing_carriers_are_contested():
    text = ("creativecommons.org/licenses/by/4.0/ ... "
            "Creative Commons Attribution-ShareAlike 3.0")
    outcome = classify(list(claims_from_text(text)))
    assert outcome["status"] == "contested"

Run the scan as a scheduled job rather than a gate. Legacy records do not change on commit, and a build that fails because a twenty-year-old record is ambiguous blocks work that has nothing to do with it. Emit the counts as a metric and the versionless list as a work queue, and the numbers will move as the queue is cleared — which is the visible progress that keeps a remediation effort funded.

What to Do With a Versionless Record

The queue is only useful if there is a decision procedure at the end of it, and there is one, in a fixed order.

The resolution procedure, in order of authorityAsk the publisher, look for a dated announcement, infer conservatively from the record date, or retire the dataset.Ask the publisherin bulk, one messageelseDated announcementrecorded as evidenceelseInfer from record datemarked as inferredelseRetire the datasetfor anything not load-bearing
Each step down this list produces a weaker answer, and the last one is often the cheapest.

Ask the publisher. For an active portal this resolves most of the queue and is the only route that produces an authoritative answer. It is worth doing in bulk: a single message listing forty datasets gets a better response than forty messages.

Look for a dated announcement. Publishers frequently announced a licensing change with a date. A record created after that date, under a portal that stated it was moving to 4.0, is reasonably resolved to 4.0 — and the announcement is the evidence to record alongside the resolution.

Infer from the record’s own date, conservatively. Where nothing better exists, the version in force when the record was created is the least unreasonable inference. Record it as inferred, with the reasoning, never as a plain identifier — and treat the dataset as carrying the more restrictive reading where the versions differ materially.

Stop using it. For a dataset that is not load-bearing, the cheapest resolution is frequently to replace it with one whose terms are clear. This option is skipped almost universally and is often the right one for a legacy layer nobody has queried in three years.

Long-Term Compliance Best Practices

  • Record the evidence string with the resolution. The substring that produced the identifier is what makes the resolution reviewable, and it costs nothing to store.
  • Keep ports in the identifier. CC-BY-3.0-DE is not CC-BY-3.0; dropping the suffix discards the pointer to the operative text.
  • Never write a resolved identifier back over the original record. Store it alongside. The legacy text is the evidence, and overwriting it destroys the ability to revisit the resolution.
  • Re-scan after every ingestion of an external catalogue. New legacy arrives constantly, usually inside a bulk import that nobody thought of as a licensing event.
  • Track the versionless count as a metric. A number that goes down is the only convincing evidence that a remediation effort is working.