# Posting Validation Results as Pull Request Comments

Post one comment and update it in place on every run, keyed by a hidden marker in the body, and put file-level detail in inline annotations rather than in the comment — a thread of forty superseded validation comments is read by nobody, and the fortieth is the only one that is true.

Reporting is where most validation efforts quietly fail. The checks work, the findings are correct, and they arrive somewhere the author does not look: a build log, an artifact, or a comment buried under the previous nineteen. This guide sits under GitHub Actions workflows for spatial data, within CI/CD Validation & Policy Enforcement for Spatial Data.

# Automated Python Implementation

The reporter below finds its own previous comment by a hidden HTML marker and edits it, so a pull request accumulates exactly one validation comment however many times the workflow runs.

#!/usr/bin/env python3
"""Post or update a single validation summary comment on a pull request."""
import json
import os
import urllib.request

MARKER = "<!-- spatial-validation-report -->"
API = "https://api.github.com"

def request(method, url, token, payload=None):
    data = json.dumps(payload).encode() if payload is not None else None
    req = urllib.request.Request(url, data=data, method=method)
    req.add_header("Authorization", f"Bearer {token}")
    req.add_header("Accept", "application/vnd.github+json")
    if data:
        req.add_header("Content-Type", "application/json")
    with urllib.request.urlopen(req, timeout=30) as response:
        return json.loads(response.read() or b"{}")

def find_existing(repo, pr_number, token):
    """The comment we posted last time, identified by the hidden marker."""
    url = f"{API}/repos/{repo}/issues/{pr_number}/comments?per_page=100"
    for comment in request("GET", url, token):
        if MARKER in (comment.get("body") or ""):
            return comment["id"]
    return None

def render(summary, findings, run_url, limit=15):
    """A summary anyone can read in five seconds, then the detail."""
    status = "passed" if not findings else "failed"
    lines = [
        MARKER,
        f"### Spatial validation {status}",
        "",
        f"{summary['checked']} file(s) checked · {summary['failed']} failing · "
        f"{summary['errored']} check error(s)",
        "",
    ]
    if findings:
        lines += ["| File | Check | Finding |", "|---|---|---|"]
        for finding in findings[:limit]:
            lines.append(
                f"| `{finding['path']}` | {finding['check']} | {finding['message']} |")
        if len(findings) > limit:
            lines.append(f"| … | | {len(findings) - limit} more, see the run log |")
        lines.append("")
    lines.append(f"[Full report]({run_url})")
    return "\n".join(lines)

def publish(repo, pr_number, token, summary, findings, run_url):
    body = render(summary, findings, run_url)
    existing = find_existing(repo, pr_number, token)
    if existing is None:
        return request("POST", f"{API}/repos/{repo}/issues/{pr_number}/comments",
                       token, {"body": body})
    return request("PATCH", f"{API}/repos/{repo}/issues/comments/{existing}",
                   token, {"body": body})

def emit_annotations(findings):
    """Workflow commands; GitHub renders these on the changed lines."""
    for finding in findings:
        location = f"file={finding['path']}"
        if finding.get("line"):
            location += f",line={finding['line']}"
        level = "error" if finding.get("severity", "error") == "error" else "warning"
        print(f"::{level} {location}::{finding['message']}")

Three decisions shape whether this reporting is used or ignored.

One comment, updated in place. The marker is what makes it findable; without it the reporter would have to guess from the body text, and a comment whose format changed would be missed and duplicated. Updating rather than appending means the comment always states the current situation.

The count comes before the table. An author who sees “3 failing” acts differently from one who sees an unbounded table, and the count is what they need first. Truncating the table at fifteen rows and linking to the full report keeps the comment readable when a bulk import goes wrong.

Annotations carry the per-file detail. They appear on the diff, next to the line in question, which is the only place a finding is guaranteed to be seen in the course of ordinary review.

# Validation and Pipeline Integration

The reporter is small and its failure modes are all about idempotence.

def test_marker_is_present_in_every_rendering():
    body = render({"checked": 2, "failed": 0, "errored": 0}, [], "https://example/run")
    assert MARKER in body

def test_passing_run_still_posts_a_summary():
    body = render({"checked": 4, "failed": 0, "errored": 0}, [], "https://example/run")
    assert "passed" in body and "4 file(s) checked" in body

def test_table_is_truncated_with_a_count():
    findings = [{"path": f"a{i}.gpkg", "check": "crs", "message": "no CRS"}
                for i in range(40)]
    body = render({"checked": 40, "failed": 40, "errored": 0}, findings, "u", limit=5)
    assert "35 more" in body

The second test encodes a judgement worth stating: a passing run should still update the comment. Leaving the previous failing summary in place after a fix is worse than having no comment at all, because the pull request then displays a failure that has been resolved.

Grant the workflow pull-requests: write and nothing else. Where changes arrive from forks, the checking job cannot have that permission, and the pattern described in GitHub Actions workflows for spatial data applies: the check uploads a report, and a separate privileged job triggered by its completion posts it.

# Writing a Finding Somebody Can Act On

A finding has three parts, and reports usually contain the first two.

What is wrong is the part everyone writes. “No CRS declared” is clear enough.

Where is usually present but often too coarse. A file path is a start; a file path plus a layer name plus a feature identifier is what turns a search into a fix. For metadata, an element path serves the same purpose. This detail exists inside the checker and is discarded on the way out surprisingly often.

What to do is almost always missing, and it is what determines whether the finding is resolved or suppressed. “No CRS declared” leaves the author to work out which CRS is expected; “no CRS declared; this repository expects EPSG:27700 or EPSG:4326” does not. The expected form is known to whoever wrote the rule and unknown to whoever hits it, and the cost of carrying it through is one field.

A fourth part is worth adding where it exists: how to reproduce locally. A finding that names the command — python validate.py roads.gpkg — lets the author iterate without pushing, which for a checker that takes seconds locally and minutes in CI is the difference between one round trip and five.

Keep the tone plain. Findings phrased as accusations get argued with; findings phrased as facts get fixed.

# Long-Term Compliance Best Practices

• Use a hidden marker, not a title match. Titles change; a marker is stable and invisible.
• Update on success as well as failure. A stale failure comment is worse than none.
• Cap the table and link to the full report. A comment listing four hundred findings is not read.
• Keep annotations and the comment in agreement. They come from the same findings list; deriving them separately is how they diverge.
• Never post from a job that has checked out untrusted code. Report from a separate privileged job triggered by the check’s completion.

# Related

• GitHub Actions Workflows for Spatial Data — the workflow and permission model this reporting sits in
• Running Spatial Validation on a Matrix of Formats — aggregating several legs into one comment
• Policy Enforcement Gates for Data PRs — the status check that blocks the merge
• Detecting Invalid Geometries with Shapely in CI — findings that benefit most from feature-level detail