Skip to content

Check re-run may evaluate the wrong pull request #307

Open

Description

@tykeal

Split out of the review of the #302 fix series, deferred to keep that PR focused.

Problem

In the check_run.rerequested handler, we resolve the pull request via:

context.payload.check_run.pull_requests[0]

check_run.pull_requests[] can legitimately contain more than one entry when a single commit is the head of several pull requests. Index 0 has no guaranteed relationship to the check run that was actually re-run, so we may evaluate the wrong PR.

This matters because different PRs can have different base branches, and the base determines the merge base and therefore the set of commits DCO evaluates. A commit set that passes against one base can fail against another, so we can publish a verdict that is simply wrong for the check the user clicked.

Why it is deferred rather than urgent

Several pull requests sharing a head SHA is uncommon in practice, and the handler itself only started working at all as part of the #305 fix (the re-run button had never worked). So this is a latent correctness gap rather than an active regression.

Suggested fix

Persist the originating pull request number in external_id when creating the check run, then read it back on re-request instead of guessing. For pre-existing check runs that have no external_id and an ambiguous pull_requests array, publish a diagnostic result rather than silently picking the first entry.

Note this touches every check we create, which is why it was kept out of the #302 series.

Related

Activity

  1. tykeal commented on Jul 30, 2026

    @tykeal
    ContributorAuthor

    Triaged: keeping this open but parked, not scheduled.

    The concern is real. check_run.pull_requests[0] has no guaranteed relationship to the check run that was actually re-run, and because the base branch determines the merge base and therefore which commits get evaluated, picking the wrong entry can publish a verdict that is wrong for the check the user clicked.

    It stays parked because the trigger is narrow — it needs one commit to be the head of several pull requests simultaneously — and because the suggested fix (persisting the pull request number in external_id at creation time and reading it back on re-request) touches every check run the app creates. That is a disproportionate blast radius for a latent gap that has not been observed in practice.

    Worth noting the handler only began working at all with the #305 fix, so there is no history of this misfiring; it is a gap that became reachable rather than a regression.

    Revisit if it is ever actually observed, or if the check-creation path is being modified for another reason and the external_id change can ride along cheaply.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions