# How to Track CC-BY-SA Attribution in Shapefiles

To track CC-BY-SA attribution in shapefiles, embed license metadata across three deterministic layers: truncated attribute columns in the .dbf, an ISO 19139-compliant .shp.xml sidecar, and a version-controlled manifest.json that maps SHA-256 hashes to attribution strings.

Because the ESRI shapefile format, finalized in 1998, has zero native fields for provenance or usage restrictions, this three-layer approach is the only reliable way to ensure compliance survives format conversion, GDAL/OGR transformations, and proprietary GIS parsers. The pattern is covered in depth under Creative Commons Licensing for GIS Datasets and fits into the broader obligations described in Geospatial Data Licensing & Compliance Fundamentals.

# Why Shapefiles Make License Tracking Hard

The ESRI shapefile specification only defines four mandatory component files: geometry (.shp), spatial index (.shx), attributes (.dbf), and projection (.prj). When publishing under CC-BY-SA 4.0 you are legally required to:

• Provide a direct link to the CC BY-SA 4.0 license text
• Credit the original creator(s) by name
• Note any modifications made to the dataset
• License all derivatives under identical terms (ShareAlike)

Without automated tracking, downstream workflows routinely strip attribution during format conversion, database ingestion, or web tile publishing. The dBASE IV format used by .dbf files limits column names to 10 characters and string values to 254 bytes — both constraints that catch engineers by surprise when they try to embed long URLs or legal notices directly.

For a broader treatment of open license obligations across dataset types, see the automated attribution mapping workflows guide, which covers ODbL, government open licenses, and multi-source lineage tracking in addition to CC licenses.

# Automated Python Implementation

The script below reads a shapefile, injects CC BY-SA 4.0 tracking fields into the .dbf (respecting the 10-character column name limit and 254-character string limit), generates a minimal ISO 19139-compliant .shp.xml sidecar, and writes a machine-readable manifest.json. It requires geopandas>=0.14, lxml>=5.1, and Python 3.10+.

import geopandas as gpd
import hashlib
import json
from pathlib import Path
from lxml import etree
from datetime import datetime, timezone


def track_cc_by_sa_attribution(
    shp_path: str,
    attribution: str,
    creator: str,
    version: str = "1.0.0",
) -> dict:
    """
    Inject CC-BY-SA 4.0 tracking into a shapefile bundle.

    Actions performed:
      1. Adds lic_type, lic_url, attr_txt columns to the .dbf
      2. Writes an ISO 19139 .shp.xml sidecar with MD_LegalConstraints
      3. Creates manifest.json with SHA-256 hashes + attribution metadata

    Returns the manifest dict so callers can log or assert against it.
    """
    base = Path(shp_path).stem
    dir_path = Path(shp_path).parent

    # ── 1. .dbf attribute injection ──────────────────────────────────────────
    # geopandas preserves all existing columns; we add three compliance fields.
    # Column names are capped at 10 chars (DBF limit); values at 254 chars.
    gdf = gpd.read_file(shp_path)
    gdf["lic_type"] = "CC BY-SA 4.0"
    gdf["lic_url"] = "creativecommons.org/licenses/by-sa/4.0/"  # 39 chars, safe
    gdf["attr_txt"] = attribution[:254]
    gdf.to_file(shp_path, driver="ESRI Shapefile")

    # ── 2. ISO 19139 .shp.xml sidecar ────────────────────────────────────────
    # ArcGIS, QGIS metadata plugins, and many catalog harvesters look for a
    # .shp.xml or .xml sidecar alongside the .shp file to populate license info.
    ns_gmd = "http://www.isotc211.org/2005/gmd"
    ns_gco = "http://www.isotc211.org/2005/gco"
    ns = {"gmd": ns_gmd, "gco": ns_gco}

    root = etree.Element(f"{{{ns_gmd}}}MD_Metadata", nsmap=ns)

    # File identifier — uniquely names this dataset for catalog deduplication
    file_id = etree.SubElement(root, f"{{{ns_gmd}}}fileIdentifier")
    etree.SubElement(file_id, f"{{{ns_gco}}}CharacterString").text = f"{base}.shp"

    # Language
    lang_el = etree.SubElement(root, f"{{{ns_gmd}}}language")
    etree.SubElement(lang_el, f"{{{ns_gco}}}CharacterString").text = "eng"

    # MD_LegalConstraints — the normative license location in ISO 19139
    constraints = etree.SubElement(root, f"{{{ns_gmd}}}resourceConstraints")
    legal = etree.SubElement(constraints, f"{{{ns_gmd}}}MD_LegalConstraints")

    use_lim = etree.SubElement(legal, f"{{{ns_gmd}}}useLimitation")
    etree.SubElement(use_lim, f"{{{ns_gco}}}CharacterString").text = (
        f"Licensed under CC BY-SA 4.0. Attribution required: {attribution}"
    )

    other_con = etree.SubElement(legal, f"{{{ns_gmd}}}otherConstraints")
    etree.SubElement(other_con, f"{{{ns_gco}}}CharacterString").text = (
        "https://creativecommons.org/licenses/by-sa/4.0/"
    )

    # Stamp the metadata creation date for audit purposes
    date_info = etree.SubElement(root, f"{{{ns_gmd}}}dateStamp")
    etree.SubElement(date_info, f"{{{ns_gco}}}DateTime").text = (
        datetime.now(timezone.utc).isoformat()
    )

    xml_path = dir_path / f"{base}.shp.xml"
    etree.ElementTree(root).write(
        xml_path, pretty_print=True, xml_declaration=True, encoding="UTF-8"
    )

    # ── 3. SHA-256 manifest ───────────────────────────────────────────────────
    # The manifest is the durable compliance artefact. It binds the attribution
    # string to cryptographic fingerprints of every file in the bundle so that
    # any post-distribution tampering or accidental field removal is detectable.
    manifest_files: dict[str, str] = {}
    for ext in (".shp", ".shx", ".dbf", ".prj", ".shp.xml"):
        candidate = dir_path / f"{base}{ext}"
        if candidate.exists():
            digest = hashlib.sha256(candidate.read_bytes()).hexdigest()
            manifest_files[candidate.name] = digest

    manifest_data = {
        "schema_version": "1.0",
        "license": "CC BY-SA 4.0",
        "license_uri": "https://creativecommons.org/licenses/by-sa/4.0/",
        "creator": creator,
        "attribution": attribution,
        "dataset_version": version,
        "generated_utc": datetime.now(timezone.utc).isoformat(),
        "files": manifest_files,
    }
    manifest_path = dir_path / "manifest.json"
    manifest_path.write_text(json.dumps(manifest_data, indent=2), encoding="utf-8")

    return manifest_data


# ── Example usage ─────────────────────────────────────────────────────────────
if __name__ == "__main__":
    result = track_cc_by_sa_attribution(
        shp_path="data/municipal_parcels.shp",
        attribution="City of Springfield Open Data Portal, 2024",
        creator="City of Springfield GIS Division",
        version="2.1.0",
    )
    print(f"Manifest written. Files hashed: {list(result['files'].keys())}")

# What each layer contributes

# Validation and Pipeline Integration

After generating the three layers, verify that attribution metadata survives your standard GIS transformations before distributing the dataset. The checks below cover the most common failure modes.

# Step 1 — Confirm .dbf field presence with ogrinfo

ogrinfo -al -so municipal_parcels.shp

Look for lic_type (String), lic_url (String), and attr_txt (String) in the field list. If they are absent, the .to_file() call may have overwritten rather than updated the layer; pass mode="a" or re-read the written file to verify round-trip integrity.

# Step 2 — Validate the ISO 19139 XML sidecar

python - <<'EOF'
from lxml import etree
tree = etree.parse("municipal_parcels.shp.xml")
ns = {
    "gmd": "http://www.isotc211.org/2005/gmd",
    "gco": "http://www.isotc211.org/2005/gco",
}
constraints = tree.findall(".//gmd:useLimitation/gco:CharacterString", ns)
assert constraints, "No useLimitation found — sidecar is missing license text"
print("Sidecar OK:", constraints[0].text[:80])
EOF

# Step 3 — Verify SHA-256 hashes post-distribution

python - <<'EOF'
import hashlib, json
from pathlib import Path

manifest = json.loads(Path("manifest.json").read_text())
errors = []
for filename, expected_hash in manifest["files"].items():
    actual = hashlib.sha256(Path(filename).read_bytes()).hexdigest()
    if actual != expected_hash:
        errors.append(f"HASH MISMATCH: {filename}")
if errors:
    raise SystemExit("\n".join(errors))
print("All file hashes verified OK")
EOF

# Step 4 — Pytest assertion for CI gates

Integrate the hash check into your test suite so that a missing or modified attribution field breaks the build. This aligns with spatial data schema linting in CI practices and can be wired into a policy enforcement gate for data PRs.

# tests/test_attribution_compliance.py
import hashlib
import json
import pytest
import geopandas as gpd
from pathlib import Path

DATASET_DIR = Path("data/municipal_parcels")

def test_dbf_has_license_columns():
    gdf = gpd.read_file(DATASET_DIR / "municipal_parcels.shp")
    assert "lic_url" in gdf.columns, "lic_url column missing from .dbf"
    assert "attr_txt" in gdf.columns, "attr_txt column missing from .dbf"
    assert gdf["lic_url"].notna().all(), "Rows with null lic_url detected"

def test_manifest_hashes_match():
    manifest = json.loads((DATASET_DIR / "manifest.json").read_text())
    for filename, expected in manifest["files"].items():
        actual = hashlib.sha256((DATASET_DIR / filename).read_bytes()).hexdigest()
        assert actual == expected, f"Hash mismatch: {filename}"

def test_manifest_has_required_fields():
    manifest = json.loads((DATASET_DIR / "manifest.json").read_text())
    for field in ("license", "license_uri", "creator", "attribution"):
        assert field in manifest, f"Manifest missing field: {field}"

Run with python -m pytest tests/test_attribution_compliance.py -v.

# Long-Term Compliance Best Practices

Shapefiles remain ubiquitous in government and municipal GIS workflows, but they are structurally ill-suited for modern metadata requirements. These practices reduce compliance drift across multi-year projects.

• Version-control manifest.json as the attribution source of truth. Commit it alongside every data release tag. If attribution needs updating (creator name change, license revision), bump the dataset_version field and recompute hashes — the diff in version control is your audit trail.
• Enforce column-naming conventions with a pre-commit hook. Use consistent prefixes (lic_, attr_, prov_) across all datasets so automated scanners can locate license fields without knowing dataset-specific schemas. A metadata schema validation and linting step in CI catches naming deviations before they reach a data portal.
• Copy the .shp.xml sidecar explicitly in every distribution script. GDAL does not automatically propagate sidecars during ogr2ogr conversions. Add an explicit shutil.copy step for any .shp.xml or LICENSE.txt file whenever your pipeline converts or packages the dataset.
• Migrate new datasets to GeoPackage. GeoPackage stores everything in a single SQLite container and supports ISO 19115-1 metadata natively via the gpkg_metadata and gpkg_metadata_reference tables. This removes the sidecar problem entirely. For an ISO 19115 metadata template generation approach compatible with GeoPackage, the same lxml-based XML generation used here applies directly.
• Pair machine-readable fields with a human-readable LICENSE.txt. The CC BY-SA 4.0 “reasonable to the medium” clause requires human-readable notices in addition to machine-readable metadata. Include a plaintext LICENSE.txt containing the full attribution string and license URI in every distributed archive.
• Run hash verification on ingest, not just at publish time. Data portals and downstream users may modify shapefiles without updating manifest.json. A lightweight ingest-time script that re-hashes the .dbf and reports any deviation catches attribution stripping before corrupted data enters a production catalog.

# Related

• Creative Commons Licensing for GIS Datasets — parent page covering CC license selection, ShareAlike obligations, and compatibility with ODbL and government open licenses
• Geospatial Data Licensing & Compliance Fundamentals — the overview page for the full compliance domain
• Automated Attribution Mapping Workflows — Python pipelines for tracking attribution across multi-source datasets including ODbL and municipal open data licenses
• Building a License Compliance Matrix for Municipal Data — constructing per-dataset compliance tables when mixing CC, ODbL, and government licenses
• ISO 19115 Metadata Template Generation — generating full ISO 19115 metadata records that carry the same license and attribution fields used in the .shp.xml sidecar