Skip to content

ci: rework the pull request template after the bug report form - #1678

Merged
Pedro Henrique Penna (ppenna) merged 1 commit into
devfrom
agents/issue-1677-pr-template
Nov 17, 2026
Merged

Pedro Henrique Penna (ppenna) merged 1 commit into
devfrom
agents/issue-1677-pr-template

Conversation

@ppenna

Copy link
Copy Markdown
Contributor

Fixes #1677

What does this pull request change?

This PR fills its own description with the new template, so this body shows how the template reads.

  • The template. .github/pull_request_template.md now has the issue's proposal: What does this pull request change?, Why is this change needed?, How was this change validated?, and Checklist, in that order. The guidance stays in HTML comments, so an unfilled template renders as empty sections plus the checklist. The conditional Changes to existing tests and OpenVMM pin sections and the CLA note sit in a closing comment. The opening comment now asks authors to redact what they paste and to report vulnerabilities privately. The template links nothing relative.

  • Where the sections are named. These now name the new sections:

    • the Pull request body row of doc/contribute.md;
    • step 6 of nvx-fix-issue and step 6 of nvx-promote-openvmm-pin;
    • the two places that described a pull request body in the old terms: the Tests rule of copilot-instructions.md and step 6 of nvx-rebase-openvmm-chain.

    Neither the template nor nvx-fix-issue tells anyone to add tests-modified any more. Both say that the Test changes workflow adds it.

  • The test. PullRequestTemplateTests in scripts/test_label_issues.py, next to BugReportFormTests, checks:

    • each heading's level, text, and visibility, in source order;
    • that the unfilled template renders empty questions and only unchecked checklist items;
    • that the Pull request body row of doc/contribute.md names the template's sections in order.

Where to start: the template, then doc/contribute.md, then the test.

Not in this change:

  • An AI-usage disclosure prompt. This was the issue's open question, and we chose to leave it out: NVX has no AI-usage policy for such a prompt to point to, and agent commits already carry the Co-authored-by: Copilot trailer. The prompt can come later with a policy.
  • CI enforcement of the sections. A pull request body is Markdown, not a form, and the issue puts enforcement out of scope.
  • A check that the template has no relative links. check_markdown_links.py would still accept a relative link that resolves in the repository but not in a pull request body.

Why is this change needed?

The old template asked only What and why and Validation. Its guidance for changed tests and OpenVMM pins sat in a trailing comment, and it told authors to add tests-modified, a label that test-changes.yml now adds and removes by itself. So each merged pull request put the missing information under a heading of its own; the issue's table lists them, from #1641 to #1672. Copilot and the maintainer read the description first, and agents fill the template through nvx-fix-issue and nvx-promote-openvmm-pin, so its prompts shape their pull requests directly.

Rejected alternatives:

  • Several templates under .github/PULL_REQUEST_TEMPLATE/. GitHub offers them only through the template query parameter, so one template keeps the conditional sections in a comment instead.
  • A new test module. CI's validate-nvx lists every scripts/test_*.py module on both operating systems, so a new module would change those lines, and the Test changes workflow flags changed lines in the actions that run the tests. The test sits next to BugReportFormTests instead, since the new template follows that form.

How was this change validated?

All checks ran locally on Windows 11 with Python 3.14. This is not a runtime change, so I used no remote host, backend, or guest.

Check Revision Result
python -m unittest scripts.test_label_issues.PullRequestTemplateTests, with the new test file in a worktree of the base 6f3faa4c565 (base) 3 of 3 failed: the old headings, no Checklist, and the contribute.md row names no section
same, on the branch 619ed422c52 3 of 3 passed
Mutations of the template: a commented #### heading; the conditional block moved to the top; a section renamed in doc/contribute.md 619ed422c52 Each one failed the tests
python -m unittest scripts.test_label_issues 619ed422c52 37 tests, OK
python scripts/local_gates.py --base origin/dev --run --only test-changes markdown-links test-layout 619ed422c52 All three passed; check_test_changes.py lists 0 changes to existing tests and 6 added items
quality-code: ruff check scripts .github/specula, compileall, pyright --pythonplatform Linux, pyright --pythonplatform Windows, ruff format --check branch All passed, with 0 Pyright errors on either platform
nvx-cli (Windows): every scripts/test_*.py module that validate-nvx lists, run without its pip install step, with Git's usr\bin on PATH and NVX_RS unset branch, before the review fix to the new test 2,259 tests OK, 91 skipped
python scripts\test_adversarial.py, python scripts\nvx.py --help same 84 tests OK, 11 skipped; exit 0

Not run locally:

  • The Docker shellcheck and shfmt steps of quality-code: from Git Bash, MSYS rewrote the container's /workspace path. This change touches no shell script.
  • The NVX_RS-gated tests: no release nvx-rs was built.

Left to CI (this Draft skips them until it is marked ready): Lint, typing, and formatting on Ubuntu, and NVX CLI / Linux and NVX CLI / Windows. local_gates.py selected no MicroVM, benchmark, or OpenVMM CI suite.

Checklist

  • Tests cover the new or fixed behavior, in the layout that
    python3 scripts/check_test_layout.py checks.
  • python3 scripts/check_test_changes.py --base origin/dev lists no change
    to an existing test, or a Changes to existing tests section below
    explains each one.
  • The documentation that the change affects is updated, including
    .github/copilot-instructions.md and the skills when a convention or
    workflow changes.
  • A change to the public API of nvx updates nvx-rs/public-api/ with
    python3 scripts/public_api.py --update.
  • Snapshots, protocols, and on-disk formats stay compatible, or the
    description says how the change versions or migrates them.

The pull request template asked only "What and why" and "Validation",
kept its guidance for changed tests and OpenVMM pins in a comment after
them, and told authors to add the tests-modified label, which the Test
changes workflow now adds and removes by itself. Merged pull requests
added what the template did not ask for under headings of their own:
what a change leaves out, its rejected alternatives, its compatibility,
the environment of each run, its changed tests, and its OpenVMM pin.

Ask the Bug report form's questions of a change instead: what it
changes, why it is needed, and how it was validated, with the backend,
host OS, host type, and guest of each run and the gates left to CI. Add
a checklist for tests, changes to existing tests, documentation, the
public API snapshots, and compatibility, each with the command that
checks it, and name the conditional "Changes to existing tests" and
"OpenVMM pin" sections in a closing comment, with the CLA note. Remind
authors to redact what they paste and to report vulnerabilities
privately. The guidance stays in HTML comments, so an unfilled template
renders as empty sections, and links nothing relative, since a pull
request body resolves no relative link to a repository file.

doc/contribute.md, copilot-instructions.md, and the nvx-fix-issue,
nvx-promote-openvmm-pin, and nvx-rebase-openvmm-chain skills name the
new sections, and say that the workflow adds tests-modified.

PullRequestTemplateTests pins the rendered sections and their order,
the commented conditional sections, and the sections that the Pull
request body row of doc/contribute.md names. All three fail at
origin/dev 6f3faa4 and pass with this change.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 17, 2026 01:29

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues were identified for the pull request template and contributor-guidance updates.

No open issues 🎉

What changed in this PR

Reworks NVX’s pull request template to mirror the bug report form, helping contributors describe changes, explain validation, and address compatibility consistently.

Changes:

  • Adds separate change, rationale, and validation questions, a checklist, and security reminders.
  • Aligns contributor documentation and agent skills with the new sections and automatic tests-modified labeling.
  • Adds tests for heading order, comment visibility, the unfilled template, and documentation consistency.
File Description
scripts/​test_label_issues.py Adds template structure and documentation consistency tests.
doc/​contribute.md Documents the new required and conditional sections.
.github/​skills/​nvx-rebase-openvmm-chain/​SKILL.md Aligns release PR guidance with the template sections.
.github/​skills/​nvx-promote-openvmm-pin/​SKILL.md Maps pin-promotion details to the new sections.
.github/​skills/​nvx-fix-issue/​SKILL.md Updates PR guidance and removes manual labeling instructions.
.github/​pull_request_template.md Adds question headings, checklist items, and security reminders.
.github/​copilot-instructions.md Names the test-change section and clarifies automatic labeling.

Review effort: Balanced

@ppenna
Pedro Henrique Penna (ppenna) marked this pull request as ready for review November 17, 2026 02:27
@ppenna
Pedro Henrique Penna (ppenna) added this pull request to the merge queue Nov 17, 2026
Pedro Henrique Penna (ppenna) added a commit that referenced this pull request Nov 17, 2026
doc/contribute.md asks every pull request to reference an issue on the
first line of its description, but nothing checks it: pull requests
such as #1675, whose only references sit in the template's HTML
comment, and #1516, whose Refs #1509 names a pull request, reference
no issue, and a reviewer has to notice.

Add the Issue link workflow, which pull_request_target runs, from the
default branch, when a pull request is opened, reopened, pushed to,
edited, labelled, or unlabelled. Its script, check_issue_link.py,
reads the pull request through gh api and passes it when GitHub links
it to an issue of the repository (closingIssuesReferences) or when
its description holds close, fix, resolve (in any form), Part of, or
Refs, with an optional colon, then #N, microsoft/nvx#N, or the
issue's URL, where N is an issue. A pull request's number, a missing
number, a bare #N, another repository's issue, and references in HTML
comments or Markdown code do not count, nor does a keyword that one
of them or the end of a block separates from its number: the script
reads quotes, lists, headings, code, and comments as CommonMark does.
It looks up at most the first 10 numbers, so a description cannot
make it call the API without bound. Pull requests by dependabot[bot]
or github-actions[bot], or labelled no-issue, are exempt, and closed
ones are left alone.

An unlinked pull request gets the needs-issue label and one marker
comment that says how to reference an issue, and the check fails;
editing the description reruns it, which then removes the label and
rewrites the comment. Only a comment of github-actions[bot] that starts
with the marker is the script's. The script does every read before its
first write, so an API error fails the check and changes nothing, and
its summary names the verdict and the numbers, never the description.

The job has no condition, since a skipped run would replace the
previous verdict, runs one check a pull request at a time, keeps no
checkout credentials, and gets only the pull request's number. Its
token reads contents and writes pull requests; reading an issue of a
public repository needs no issues permission, so the nightly report
and issue labelling stay the only workflows that write issues.

Evidence: dry runs against the live repository pass #1678, which
GitHub links to #1677, flag #1675, #1516, and #1509, as reading
their descriptions confirms, and leave merged pull requests such
as #1672 alone; before they merged, they passed #1674, which
GitHub linked to #1666, and flagged #1671.

test_issue_link.py covers the parser, with indented code, fences that
end with their container, and hidden text or a block end between a
keyword and its number; the verdicts and the writes through a fake gh
api, the bound on lookups, an API error at each read, a comment that
quotes the marker, and the workflow's triggers, permissions, checkout,
concurrency, and inputs. validate-nvx runs it on Linux and Windows.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Merged via the queue into dev with commit e5330ad Nov 17, 2026
86 checks passed
@ppenna
Pedro Henrique Penna (ppenna) deleted the agents/issue-1677-pr-template branch November 17, 2026 02:57
Pedro Henrique Penna (ppenna) added a commit that referenced this pull request Nov 17, 2026
doc/contribute.md asks every pull request to reference an issue on the
first line of its description, but nothing checks it: pull requests
such as #1675, whose only references sit in the template's HTML
comment, and #1516, whose Refs #1509 names a pull request, reference
no issue, and a reviewer has to notice.

Add the Issue link workflow, which pull_request_target runs, from the
default branch, when a pull request is opened, reopened, pushed to,
edited, labelled, or unlabelled. Its script, check_issue_link.py,
reads the pull request through gh api and passes it when GitHub links
it to an issue of the repository (closingIssuesReferences) or when
its description holds close, fix, resolve (in any form), Part of, or
Refs, with an optional colon, then #N, microsoft/nvx#N, or the
issue's URL, where N is an issue. A pull request's number, a missing
number, a bare #N, another repository's issue, and references in HTML
comments or Markdown code do not count, nor does a keyword that one
of them or the end of a block separates from its number: the script
reads quotes, lists, headings, code, and comments as CommonMark does.
It looks up at most the first 10 numbers, so a description cannot
make it call the API without bound. Pull requests by dependabot[bot]
or github-actions[bot], or labelled no-issue, are exempt, and closed
ones are left alone.

An unlinked pull request gets the needs-issue label and one marker
comment that says how to reference an issue, and the check fails;
editing the description reruns it, which then removes the label and
rewrites the comment. Only a comment of github-actions[bot] that starts
with the marker is the script's. The script does every read before its
first write, so an API error fails the check and changes nothing, and
its summary names the verdict and the numbers, never the description.

The job has no condition, since a skipped run would replace the
previous verdict, runs one check a pull request at a time, keeps no
checkout credentials, and gets only the pull request's number. Its
token reads contents and writes pull requests; reading an issue of a
public repository needs no issues permission, so the nightly report
and issue labelling stay the only workflows that write issues.

Evidence: dry runs against the live repository pass #1678, which
GitHub links to #1677, flag #1675, #1516, and #1509, as reading
their descriptions confirms, and leave merged pull requests such
as #1672 alone; before they merged, they passed #1674, which
GitHub linked to #1666, and flagged #1671.

test_issue_link.py covers the parser, with indented code, fences that
end with their container, and hidden text or a block end between a
keyword and its number; the verdicts and the writes through a fake gh
api, the bound on lookups, an API error at each read, a comment that
quotes the marker, and the workflow's triggers, permissions, checkout,
concurrency, and inputs. validate-nvx runs it on Linux and Windows.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pedro Henrique Penna (ppenna) added a commit that referenced this pull request Nov 17, 2026
doc/contribute.md asks every pull request to reference an issue on the
first line of its description, but nothing checks it: pull requests
such as #1675, whose only references sit in the template's HTML
comment, and #1516, whose Refs #1509 names a pull request, reference
no issue, and a reviewer has to notice.

Add the Issue link workflow, which pull_request_target runs, from the
default branch, when a pull request is opened, reopened, pushed to,
edited, labelled, or unlabelled. Its script, check_issue_link.py,
reads the pull request through gh api and passes it when GitHub links
it to an issue of the repository (closingIssuesReferences) or when
its description holds close, fix, resolve (in any form), Part of, or
Refs, with an optional colon, then #N, microsoft/nvx#N, or the
issue's URL, where N is an issue. A pull request's number, a missing
number, a bare #N, another repository's issue, and references in HTML
comments or Markdown code do not count, nor does a keyword that one
of them or the end of a block separates from its number: the script
reads quotes, lists, headings, code, and comments as CommonMark does,
and lets a comment run across blocks, as a browser does, until raw
HTML ends it. It looks up at most the first 10 numbers, so a
description cannot make it call the API without bound. Pull requests
by dependabot[bot] or github-actions[bot], or labelled no-issue, are
exempt, and closed ones are left alone.

An unlinked pull request gets the needs-issue label and one marker
comment that says how to reference an issue, and the check fails;
editing the description reruns it, which then removes the label and
rewrites the comment. Only a comment of github-actions[bot] that starts
with the marker is the script's. The script does every read before its
first write, so an API error fails the check and changes nothing, and
its summary names the verdict and the numbers, never the description.

The job has no condition, since a skipped run would replace the
previous verdict, runs one check a pull request at a time, keeps no
checkout credentials, and gets only the pull request's number. Its
token reads contents and writes pull requests; reading an issue of a
public repository needs no issues permission, so the nightly report
and issue labelling stay the only workflows that write issues.

Evidence: dry runs against the live repository pass #1678, which
GitHub links to #1677, flag #1675, #1516, and #1509, as reading
their descriptions confirms, and leave merged pull requests such
as #1672 alone; before they merged, they passed #1674, which
GitHub linked to #1666, and flagged #1671. Each parser case also
agrees with what GitHub's Markdown API renders, apart from a
reference that a comment splits and a pull request's URL.

test_issue_link.py covers the parser, with indented code, fences that
end with their container, setext headings, sibling list items,
comments that outlast their block or end as a browser ends them, and
hidden text or a block end between a keyword and its number; the
verdicts and the writes through a fake gh api, the bound on lookups,
an API error at each read, a comment that quotes the marker or has no
author, and the workflow's triggers, permissions, checkout,
concurrency, and inputs. validate-nvx runs it on Linux and Windows.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pedro Henrique Penna (ppenna) added a commit that referenced this pull request Nov 17, 2026
doc/contribute.md asks every pull request to reference an issue on the
first line of its description, but nothing checks it: pull requests
such as #1675, whose only references sit in the template's HTML
comment, and #1516, whose Refs #1509 names a pull request, reference
no issue, and a reviewer has to notice.

Add the Issue link workflow, which pull_request_target runs, from the
default branch, when a pull request is opened, reopened, pushed to,
edited, labelled, or unlabelled. Its script, check_issue_link.py,
reads the pull request through gh api and passes it when GitHub links
it to an issue of the repository (closingIssuesReferences) or when
its description holds close, fix, resolve (in any form), Part of, or
Refs, with an optional colon, then #N, microsoft/nvx#N, or the
issue's URL, where N is an issue. A pull request's number, a missing
number, a bare #N, another repository's issue, and references in HTML
comments or Markdown code do not count, nor does a keyword that one
of them or the end of a block separates from its number: the script
reads quotes, lists, headings, code, and comments as CommonMark does,
and lets a comment run across blocks, as a browser does, until raw
HTML ends it. It looks up at most the first 10 numbers, so a
description cannot make it call the API without bound. Pull requests
by dependabot[bot] or github-actions[bot], or labelled no-issue, are
exempt, and closed ones are left alone.

An unlinked pull request gets the needs-issue label and one marker
comment that says how to reference an issue, and the check fails;
editing the description reruns it, which then removes the label and
rewrites the comment. Only a comment of github-actions[bot] that starts
with the marker is the script's. The script does every read before its
first write, so an API error fails the check and changes nothing, and
its summary names the verdict and the numbers, never the description.

The job has no condition, since a skipped run would replace the
previous verdict, runs one check a pull request at a time, keeps no
checkout credentials, and gets only the pull request's number. Its
token reads contents and writes pull requests; reading an issue of a
public repository needs no issues permission, so the nightly report
and issue labelling stay the only workflows that write issues.

Evidence: dry runs against the live repository pass #1678, which
GitHub links to #1677, flag #1675, #1516, and #1509, as reading
their descriptions confirms, and leave merged pull requests such
as #1672 alone; before they merged, they passed #1674, which
GitHub linked to #1666, and flagged #1671. Each parser case also
agrees with what GitHub's Markdown API renders, apart from a
reference that a comment splits and a pull request's URL.

test_issue_link.py covers the parser, with indented code, fences that
end with their container, setext headings, sibling list items,
comments that outlast their block or end as a browser ends them, and
hidden text or a block end between a keyword and its number; the
verdicts and the writes through a fake gh api, the bound on lookups,
an API error at each read, a comment that quotes the marker or has no
author, and the workflow's triggers, permissions, checkout,
concurrency, and inputs. validate-nvx runs it on Linux and Windows.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pedro Henrique Penna (ppenna) added a commit that referenced this pull request Nov 17, 2026
doc/contribute.md asks every pull request to reference an issue on the
first line of its description, but nothing checks it: pull requests
such as #1675, whose only references sit in the template's HTML
comment, and #1516, whose Refs #1509 names a pull request, reference
no issue, and a reviewer has to notice.

Add the Issue link workflow, which pull_request_target runs, from the
default branch, when a pull request is opened, reopened, pushed to,
edited, labelled, or unlabelled. Its script, check_issue_link.py,
reads the pull request through gh api, in UTF-8 whatever the locale,
and passes it when GitHub links it to an issue of the repository
(closingIssuesReferences) or when its description holds close, fix,
resolve (in any form), Part of, or Refs, with an optional colon,
then #N, microsoft/nvx#N, or the issue's URL, where N is an issue. A
pull request's number, a missing number, a bare #N, another
repository's issue, and references in HTML comments or Markdown code
do not count, nor does a keyword that one of them or the end of a
block separates from its number: the script reads quotes, lists,
headings, code, and HTML blocks as GitHub's CommonMark does, reads
raw HTML as text but for its comments, and lets a comment run across
blocks, as a browser does, until raw HTML ends it. It looks up at
most the first 10 numbers, so a description cannot make it call the
API without bound. Pull requests by dependabot[bot] or
github-actions[bot], or labelled no-issue, are exempt, and closed
ones are left alone.

An unlinked pull request gets the needs-issue label and one marker
comment that says how to reference an issue, and the check fails;
editing the description reruns it, which then removes the label and
rewrites the comment. Only a comment of github-actions[bot] that starts
with the marker is the script's. The script does every read before its
first write, so an API error fails the check and changes nothing, and
its summary names the verdict and the numbers, never the description.

The job has no condition, since a skipped run would replace the
previous verdict, runs one check a pull request at a time, keeps no
checkout credentials, and gets only the pull request's number. Its
token reads contents and writes pull requests; reading an issue of a
public repository needs no issues permission, so the nightly report
and issue labelling stay the only workflows that write issues.

Evidence: dry runs against the live repository pass #1678, which
GitHub links to #1677, flag #1675, #1516, and #1509, as reading
their descriptions confirms, and leave merged pull requests such
as #1672 alone; before they merged, they passed #1674, which
GitHub linked to #1666, and flagged #1671. Each parser case also
agrees with what GitHub's Markdown API renders, apart from a
reference that a comment splits, a pull request's URL, and the text
of a declaration, which GitHub hides.

test_issue_link.py covers the parser, with indented code, fences that
end with their container, setext headings, sibling and empty list
items, HTML blocks, comments that outlast their block or end as a
browser ends them, and hidden text or a block end between a keyword
and its number; the verdicts and the writes through a fake gh api,
gh's input and output in UTF-8, the bound on lookups, an API error at
each read, a comment that quotes the marker or has no author, and the
workflow's triggers, permissions, checkout, concurrency, and inputs.
validate-nvx runs it on Linux and Windows.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci: rework the pull request template after the bug report form

2 participants