# FGDC to ISO 19115 Conversion Pipelines

Migrating legacy geospatial metadata from the Federal Geographic Data Committee (FGDC) Content Standard for Digital Geospatial Metadata (CSDGM) to the international ISO 19115/19139 standard is a foundational requirement for modern data portals, cross-agency interoperability, and automated cataloging. Manual translation is unsustainable at scale: a single agency archive can hold thousands of FGDC records accumulated over decades, each with idiosyncratic date formats, compound contact blocks, and ad-hoc bounding-box encodings. This page is part of the broader Automated Metadata Generation & Schema Mapping discipline, where programmatic transformation replaces error-prone manual editing and enables seamless publishing to enterprise catalogs and open data platforms.

# Prerequisites

Before implementing the conversion pipeline, satisfy these requirements:

1. Python 3.10+ — required for match/case in crosswalk dispatch logic
2. lxml>=4.9 — XML parsing, XPath traversal, and ISO 19139 serialisation
3. xmlschema>=2.4 — XSD validation against ISO 19139 schemas
4. pyproj>=3.6 — CRS lookup, EPSG resolution, and WKT conversion
5. pyyaml>=6.0 — loading the declarative crosswalk registry from YAML
6. FGDC CSDGM v2.0 XSD — obtain from the FGDC standards repository (no external link; mirror locally)
7. ISO 19139 XSD bundle — gmd, gco, gss, gts namespace schemas mirrored locally for air-gapped validation
8. Environment variable ISO19139_XSD_DIR — path to the locally mirrored XSD bundle, consumed by the validator
9. Write access to a staging directory — the pipeline writes to .tmp then renames for atomic output

# Pipeline Architecture

The diagram below illustrates the six-stage ETL flow. Each stage is stateless and independently testable. Rejection paths log structured JSON entries for operational review.

# Concept & Spec Reference

# Standards overview

FGDC CSDGM (Content Standard for Digital Geospatial Metadata) — a U.S. federal standard originally published in 1994 and revised in 1998. It uses a flat, element-centric XML model without XML namespaces. Core sections include idinfo, dataqual, spdoinfo, spref, eainfo, distinfo, and metainfo.

ISO 19115-1:2014 / ISO 19115-2:2019 — the international abstract model for geospatial metadata. The serialisation format is ISO 19139:2007 (XML encoding). It uses explicit XML namespaces: gmd (http://www.isotc211.org/2005/gmd), gco (http://www.isotc211.org/2005/gco), and several auxiliary namespaces (gss, gts, gsr).

# Field-by-field mapping table

# Implementation Walkthrough

# Step 1 — Ingest & normalise

Load the FGDC file with strict parsing disabled for recovery mode only during triage; use recover=False for production pipelines to surface broken inputs early.

import os
from lxml import etree

def load_fgdc(path: str) -> etree._Element:
    """Parse FGDC CSDGM XML with strict settings; raise on malformed input."""
    parser = etree.XMLParser(
        resolve_entities=False,  # prevent XXE
        recover=False,           # fail fast on malformed XML
        remove_blank_text=True,
    )
    with open(path, "rb") as fh:
        raw = fh.read().lstrip(b"\xef\xbb\xbf")  # strip UTF-8 BOM
    return etree.fromstring(raw, parser)

# Step 2 — Extract & flatten

Decouple XPath extraction from mapping logic. Return a plain dictionary so the crosswalk registry operates on Python types, not XML nodes.

def extract_fgdc(root: etree._Element) -> dict:
    """Flatten key FGDC sections into a Python dict for downstream mapping."""
    def txt(xpath: str) -> str:
        nodes = root.xpath(xpath)
        return nodes[0].text.strip() if nodes and nodes[0].text else ""

    def multi(xpath: str) -> list[str]:
        return [n.text.strip() for n in root.xpath(xpath) if n.text]

    return {
        "title":       txt("idinfo/citation/citeinfo/title"),
        "pubdate":     txt("idinfo/citation/citeinfo/pubdate"),
        "abstract":    txt("idinfo/descript/abstract"),
        "purpose":     txt("idinfo/descript/purpose"),
        "westbc":      txt("idinfo/spdom/bounding/westbc"),
        "eastbc":      txt("idinfo/spdom/bounding/eastbc"),
        "northbc":     txt("idinfo/spdom/bounding/northbc"),
        "southbc":     txt("idinfo/spdom/bounding/southbc"),
        "theme_keys":  multi("idinfo/keywords/theme/themekey"),
        "theme_kt":    txt("idinfo/keywords/theme/themekt"),
        "place_keys":  multi("idinfo/keywords/place/placekey"),
        "proc_steps":  multi("dataqual/lineage/procstep/procdesc"),
        "contact_org": txt("idinfo/ptcontac/cntinfo/cntorgp/cntorg"),
        "contact_per": txt("idinfo/ptcontac/cntinfo/cntorgp/cntper"),
        "metd":        txt("metainfo/metd"),
        "distrib_fmt": txt("distinfo/stdorder/digform/digtinfo/formname"),
    }

# Step 3 — Normalise dates

FGDC date strings appear as YYYYMMDD, YYYY-MM-DD, YYYYMM, or bare YYYY. Normalise before injecting into ISO elements.

import re
from datetime import date

def normalise_fgdc_date(raw: str) -> str:
    """Return an ISO 8601 date string from a FGDC pubdate value."""
    raw = re.sub(r"[^0-9]", "", raw)
    if len(raw) == 8:
        return f"{raw[:4]}-{raw[4:6]}-{raw[6:8]}"
    if len(raw) == 6:
        return f"{raw[:4]}-{raw[4:6]}-01"
    if len(raw) == 4:
        return f"{raw}-01-01"
    return str(date.today())  # last-resort default; caller should log a warning

# Step 4 — Map & transform

Use lxml.builder to construct the ISO 19139 tree. The snippet below shows the citation block; extend the same pattern for each section.

from lxml.builder import ElementMaker

GMD = "http://www.isotc211.org/2005/gmd"
GCO = "http://www.isotc211.org/2005/gco"

E_gmd = ElementMaker(namespace=GMD, nsmap={"gmd": GMD, "gco": GCO})
E_gco = ElementMaker(namespace=GCO, nsmap={"gmd": GMD, "gco": GCO})

def build_citation(data: dict) -> object:
    """Construct a CI_Citation element from extracted FGDC data."""
    return E_gmd.citation(
        E_gmd.CI_Citation(
            E_gmd.title(E_gco.CharacterString(data["title"])),
            E_gmd.date(
                E_gmd.CI_Date(
                    E_gmd.date(E_gco.Date(normalise_fgdc_date(data["pubdate"]))),
                    E_gmd.dateType(
                        E_gmd.CI_DateTypeCode(
                            "publication",
                            codeList="http://standards.iso.org/ittf/PubliclyAvailableStandards/ISO_19139_Schemas/resources/codelist/gmxCodelists.xml#CI_DateTypeCode",
                            codeListValue="publication",
                        )
                    ),
                )
            ),
        )
    )

# Step 5 — Enrich mandatory ISO fields

ISO 19115 requires several elements that FGDC does not define:

import uuid

def build_metadata_root(data: dict) -> object:
    """Assemble the MD_Metadata root with mandatory fields defaulted."""
    file_id = str(uuid.uuid4())
    return E_gmd.MD_Metadata(
        E_gmd.fileIdentifier(E_gco.CharacterString(file_id)),
        E_gmd.language(E_gco.LanguageCode("eng", codeList="", codeListValue="eng")),
        E_gmd.characterSet(
            E_gmd.MD_CharacterSetCode(
                "utf8",
                codeList="http://standards.iso.org/ittf/PubliclyAvailableStandards/ISO_19139_Schemas/resources/codelist/gmxCodelists.xml#MD_CharacterSetCode",
                codeListValue="utf8",
            )
        ),
        E_gmd.hierarchyLevel(
            E_gmd.MD_ScopeCode(
                "dataset",
                codeList="http://standards.iso.org/ittf/PubliclyAvailableStandards/ISO_19139_Schemas/resources/codelist/gmxCodelists.xml#MD_ScopeCode",
                codeListValue="dataset",
            )
        ),
        build_citation(data),
        # ... remaining sections
    )

# Step 6 — Serialise & validate

Write the tree to a byte string and validate against the ISO 19139 XSD bundle before committing to disk. For XSD validation patterns in depth, see validating FGDC metadata against XML schemas.

import xmlschema
import os
import pathlib

def serialise_and_validate(root_elem: object, output_path: str) -> bool:
    """Serialise ISO 19139 XML and validate against the local XSD bundle."""
    xsd_dir = os.environ["ISO19139_XSD_DIR"]
    schema = xmlschema.XMLSchema(str(pathlib.Path(xsd_dir) / "gmd" / "gmd.xsd"))

    xml_bytes = etree.tostring(root_elem, pretty_print=True, xml_declaration=True, encoding="UTF-8")

    errors = list(schema.iter_errors(xml_bytes))
    if errors:
        for err in errors:
            print(f"XSD error: {err}")
        return False

    tmp_path = output_path + ".tmp"
    with open(tmp_path, "wb") as fh:
        fh.write(xml_bytes)
    os.replace(tmp_path, output_path)  # atomic rename
    return True

# Validation & CI Integration

Integrate the pipeline as a pre-commit hook or GitHub Actions step so invalid outputs never reach the catalog. A minimal CI assertion pattern:

# tests/test_fgdc_conversion.py
import pathlib
import pytest
from your_package.converter import load_fgdc, extract_fgdc, build_metadata_root, serialise_and_validate

FIXTURE_DIR = pathlib.Path(__file__).parent / "fixtures"

@pytest.mark.parametrize("fgdc_file", FIXTURE_DIR.glob("*.xml"))
def test_roundtrip_validates(fgdc_file, tmp_path):
    root = load_fgdc(str(fgdc_file))
    data = extract_fgdc(root)
    iso_root = build_metadata_root(data)
    out = str(tmp_path / fgdc_file.with_suffix(".iso.xml").name)
    assert serialise_and_validate(iso_root, out), f"XSD validation failed for {fgdc_file.name}"

Run in CI with python -m pytest tests/ -v --tb=short. Pin xmlschema and lxml in requirements-dev.txt to prevent schema resolution regressions on minor library updates.

For broader spatial schema linting across formats, see metadata schema validation and linting.

# Derivative & Lineage Management

Conversions are themselves a transformation step and must be captured in the ISO lineage chain. Every output record should carry:

• An LI_ProcessStep describing the conversion software version, operator, and date
• The source FGDC filename and its SHA-256 checksum as LI_Source references
• A CI_Date stamped at conversion time, distinct from the original FGDC pubdate

When downstream operations such as reprojection, attribute join, or format conversion are applied to a dataset whose metadata was produced by this pipeline, append additional LI_ProcessStep elements rather than overwriting existing steps. This preserves the full audit trail expected by automated metadata generation & schema mapping workflows.

If the converted records feed DCAT-AP spatial profile mapping, preserve the fileIdentifier UUID so that catalog systems can correlate the DCAT dcat:Dataset with the originating ISO record.

For batch archives where hundreds of datasets share a common lineage event (e.g., a bulk FGDC export from a legacy GIS), use a single LI_Source referencing the parent archive and reference it from each output record rather than duplicating the provenance text.

# Pitfalls & Resolution Table

# Related

• Automated Metadata Generation & Schema Mapping — parent topic covering the full metadata automation landscape
• Validating FGDC Metadata Against XML Schemas — deep dive into XSD, Schematron, and CI gate patterns for FGDC and ISO 19139 outputs
• ISO 19115 Metadata Template Generation — reusable scaffolds and boilerplate enforcement for consistent ISO header structures
• DCAT-AP Spatial Profile Mapping — crosswalk compatibility with European and federal open data frameworks
• Metadata Schema Validation and Linting — CI integration patterns for spatial schema linting across formats