Geospatial Compliance Metadata Automation Hub

Building HTML Audit Summaries from Metadata Records

To build an HTML audit summary from spatial metadata records, iterate the parsed records, compute the set of mandatory fields that are missing or empty in each one, and render the results into an autoescaped jinja2 HTML table that flags every gap. The output is a fast, human-readable page a data steward can scan to see, at a glance, which records in the inventory are incomplete and exactly which elements they lack.

This is the HTML-first companion to the PDF workflow in Automated Compliance Report Generation, and it belongs to the wider Spatial Data Audit Reporting & Compliance Governance domain, where the summary is the everyday internal-review artifact that sits upstream of the formal, signed reports. Where a PDF is built for submission, an HTML summary is built for the daily loop of spotting and fixing incomplete metadata before it reaches a formal report.

Automated Python Implementation

The script below is self-contained and runnable. Given a list of parsed metadata records — plain dictionaries of the kind produced by an ISO 19115 or DCAT-AP parser — it checks each record against a mandatory-field list, records which fields are missing, computes a per-record status, and renders an HTML page with a summary line and a per-record table. Missing fields are listed explicitly in their own column. The only dependency is jinja2.

#!/usr/bin/env python3
"""Render an HTML audit summary from parsed spatial metadata records.

Usage:
    python build_html_audit.py

Dependency:
    pip install "jinja2>=3.1"
"""
from __future__ import annotations

from dataclasses import dataclass, field
from datetime import datetime, timezone

from jinja2 import DictLoader, select_autoescape
from jinja2.sandbox import SandboxedEnvironment

# The mandatory metadata elements every record must carry to be complete.
MANDATORY_FIELDS = ("title", "abstract", "bbox", "crs", "license", "contact")


@dataclass
class AuditedRecord:
    """A metadata record paired with the outcome of its completeness audit."""
    record_id: str
    values: dict
    missing: list[str] = field(default_factory=list)

    @property
    def status(self) -> str:
        """'complete' when nothing is missing, otherwise 'incomplete'."""
        return "complete" if not self.missing else "incomplete"


def audit_record(record_id: str, values: dict) -> AuditedRecord:
    """Return an AuditedRecord flagging any missing mandatory fields.

    A field counts as missing when its key is absent or its value is
    empty (None, empty string, or whitespace only).
    """
    missing = []
    for name in MANDATORY_FIELDS:
        value = values.get(name)
        if value is None or (isinstance(value, str) and not value.strip()):
            missing.append(name)
    return AuditedRecord(record_id=record_id, values=values, missing=missing)


AUDIT_TEMPLATE = """<!doctype html>
<html lang="en"><head><meta charset="utf-8">
<title>Metadata Audit Summary</title></head>
<body>
<h1>Metadata Audit Summary</h1>
<p>Generated: {{ generated_at }}</p>
<p>{{ records | length }} records &middot;
   {{ complete_count }} complete &middot;
   {{ incomplete_count }} incomplete</p>
<table border="1" cellspacing="0" cellpadding="4">
  <thead>
    <tr><th>Record</th><th>Title</th><th>Status</th><th>Missing fields</th></tr>
  </thead>
  <tbody>
  {% for rec in records %}
    <tr>
      <td>{{ rec.record_id }}</td>
      <td>{{ rec.values.get("title") or "—" }}</td>
      <td>{{ rec.status }}</td>
      <td>{% if rec.missing %}{{ rec.missing | join(", ") }}{% else %}—{% endif %}</td>
    </tr>
  {% endfor %}
  </tbody>
</table>
</body></html>
"""


def render_audit_html(records: list[AuditedRecord]) -> str:
    """Render audited records to an autoescaped HTML summary string."""
    env = SandboxedEnvironment(
        loader=DictLoader({"audit.html": AUDIT_TEMPLATE}),
        autoescape=select_autoescape(["html"]),
    )
    template = env.get_template("audit.html")
    complete = sum(1 for r in records if r.status == "complete")
    return template.render(
        records=records,
        generated_at=datetime.now(timezone.utc).isoformat(),
        complete_count=complete,
        incomplete_count=len(records) - complete,
    )


def build_summary(raw_records: dict[str, dict], out_path: str) -> str:
    """Audit every raw record, render the HTML summary, and write it out."""
    audited = [audit_record(rid, vals) for rid, vals in sorted(raw_records.items())]
    html = render_audit_html(audited)
    with open(out_path, "w", encoding="utf-8") as handle:
        handle.write(html)
    return out_path


if __name__ == "__main__":
    # Records as they might arrive from an ISO 19115 or DCAT-AP parser.
    parsed = {
        "parcels_2025": {
            "title": "Cadastral Parcels 2025",
            "abstract": "Parcel boundaries for the metropolitan district.",
            "bbox": "POLYGON((...))",
            "crs": "EPSG:4326",
            "license": "https://creativecommons.org/licenses/by/4.0/",
            "contact": "gis@example.gov",
        },
        "flood_zones_v4": {
            "title": "Flood Zones v4",
            "abstract": "Modelled 1-in-100-year flood extent.",
            "bbox": "POLYGON((...))",
            "crs": "EPSG:4326",
            "license": "",                # empty -> flagged as missing
            "contact": "hydro@example.gov",
        },
        "roads_centreline": {
            "title": "Roads <centreline>",   # angle brackets exercise autoescape
            "abstract": "Road centrelines.",
            "bbox": "POLYGON((...))",
            # crs and contact keys absent -> flagged as missing
            "license": "https://opendatacommons.org/licenses/odbl/1-0/",
        },
    }
    written = build_summary(parsed, out_path="metadata_audit_summary.html")
    print(f"Wrote {written}")

Running the script writes metadata_audit_summary.html. The flood_zones_v4 record is flagged for an empty license, and roads_centreline is flagged for the absent crs and contact keys — while its deliberately mangled title Roads <centreline> is rendered as literal text because autoescaping converts the angle brackets to entities rather than letting them become markup. The summary line at the top gives the reviewer an immediate complete-versus-incomplete count for the whole inventory.

Validation and pipeline integration

Test the audit logic and the rendering separately: the completeness check is pure and easy to assert, and the render step needs only a smoke test that confirms flags and escaping reach the output.

# tests/test_html_audit.py
from build_html_audit import audit_record, render_audit_html, build_summary


def test_missing_and_empty_fields_are_flagged():
    rec = audit_record("r1", {"title": "T", "abstract": "A", "bbox": "B",
                              "crs": "EPSG:4326", "license": "  ", "contact": None})
    assert set(rec.missing) == {"license", "contact"}
    assert rec.status == "incomplete"


def test_complete_record_has_no_findings():
    vals = {f: "x" for f in ("title", "abstract", "bbox",
                             "crs", "license", "contact")}
    rec = audit_record("r2", vals)
    assert rec.missing == []
    assert rec.status == "complete"


def test_render_escapes_and_flags(tmp_path):
    out = tmp_path / "audit.html"
    build_summary({"r3": {"title": "A <b> title"}}, str(out))
    html = out.read_text(encoding="utf-8")
    assert "A &lt;b&gt; title" in html      # autoescape neutralised the markup
    assert "incomplete" in html             # missing fields produced a finding

Run the generator and its tests locally, then let CI produce the summary from a fixture on every change:

# Install the pinned dependency and run the checks
pip install "jinja2>=3.1" pytest

pytest tests/test_html_audit.py -q

# Produce a sample summary from the module's __main__ block
python build_html_audit.py
name: Build Metadata Audit Summary

on:
  pull_request:
    paths:
      - 'reporting/**'

jobs:
  build-summary:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.11'
      - run: pip install "jinja2>=3.1" pytest
      - run: pytest tests/test_html_audit.py -q
      - run: python reporting/build_html_audit.py
      - uses: actions/upload-artifact@v4
        with:
          name: metadata-audit-summary
          path: metadata_audit_summary.html

Long-term compliance best practices

  • Keep the mandatory-field list in one place. Define MANDATORY_FIELDS once and share it between the audit summary and the upstream validators, so the HTML report and the enforcing gate never disagree about what “complete” means. The list itself belongs to your metadata schema validation and linting rules.
  • Treat empty as missing. A key that is present but holds an empty string or whitespace is a common failure mode from XML parsers that emit empty elements. Auditing for non-empty values, not just key presence, catches records that look complete but carry nothing usable.
  • Always autoescape. Metadata drawn from source XML routinely contains ampersands and angle brackets. A SandboxedEnvironment with select_autoescape guarantees those render as text and cannot turn a malformed record into markup injection in the report.
  • Render from captured audit results, not live re-checks. Feed the summary the audit outcomes recorded when the pipeline ran so the page reflects a point in time. Re-auditing at render time reports the present condition of the data instead of the condition on the date the summary claims to describe.
  • Sort records deterministically. Rendering from a sorted record set means the same inventory always produces the same page, which makes two summaries diffable and lets a reviewer spot exactly what changed between runs.
  • Link each incomplete record to its fix path. Extend the missing-fields column with the record’s source location so a steward can jump straight from the finding to the file that needs editing, shortening the audit-to-remediation loop.