Skip to content

Latest commit

 

History

History

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
 
 

README.md

Pull Request Dashboard

A centralized shared workflow that builds and publishes a per-repository pull request triage dashboard for opted-in OpenTelemetry repositories. For each target repository, the workflow scans open PRs, classifies who has the next action, publishes the dashboard as a GitHub issue, and optionally sends Slack notifications.

The workflow runs from open-telemetry/shared-workflows and targets the repositories listed in repositories.json. Target repositories do not need to host any workflow files.

Webhook-triggered incremental runs keep active dashboards close to real time. Hourly backfill runs provide a backstop for missed or failed targeted refreshes.

The classification cache reuses prior results for unchanged review threads, minimizing Copilot token usage.

Webhook queue

Webhook events are coalesced by repository and pull request before they start GitHub Actions. One drain workflow claims a bounded batch and processes up to four repositories concurrently on one runner; pull requests from the same repository remain sequential. An event received while its pull request is being processed marks that item dirty and schedules one follow-up pass.

New draft pull requests bypass the queue for their first targeted refresh so the dashboard status comment is created near the pull request description. Later events use the configured queue mode. The hourly publisher checks a bounded round-robin batch of open drafts and repairs only missing managed comments.

Manual targeted runs and hourly backfills bypass the queue. Queue leases recover work after interrupted drain runs, and the hourly backfill remains the final correctness backstop. Canary and all-mode webhooks do not fall back to direct workflow dispatch when queue dependencies fail, because that would recreate the runner burst the queue is designed to prevent. The next hourly backfill repairs any refresh that could not be persisted.

Dashboard columns

The dashboard groups open non-draft pull requests by who is expected to act next (e.g. Waiting on reviewers, Waiting on authors, Waiting on maintainers). Draft PRs are listed separately at the bottom unless large_repo rendering is enabled. Within each group, rows are sorted longest-waiting first. Every row has these six columns:

  • PR — Pull request number and title, followed by any configured matching labels. The number autolinks to the PR on GitHub. Configured labels are rendered inline for both active and draft PRs.
  • Author — GitHub login of the PR author.
  • Reviewers — Reviewers who have engaged with the PR, each annotated with one or more icons:
    • ✅ active approval
    • ✔️ active approval (non-code-owner — does not count toward required_approvals)
    • 💬 unresolved feedback from reviewer
    • 📌 has tracked top-level feedback that still needs author action
    • 🔴 requested changes
    • ⏳ a review is in flight (a human reviewer was requested again after reviewing, or a Copilot review is pending while it holds the PR)
    • Icons combine when multiple states apply. For example, 💬📌 means the reviewer has unresolved inline feedback and top-level feedback that still needs author action; ✅ may accompany either or both.
  • CI — Aggregate check status across the PR's required status checks. Optional checks do not affect this column:
    • ✅ all required checks passing
    • ⏳ at least one required check pending, none failing
    • ❌ at least one required check failing
    • ? check data could not be fetched
  • Conflicts — Whether the PR has merge conflicts against its base branch:
    • ✅ no conflicts
    • ❌ has conflicts
    • ? could not be determined
  • Age — How long the PR has been waiting on the current next-action owner, not the calendar age of the PR. For example, a six-month-old PR that received a reviewer comment yesterday shows 1d when it is waiting on the author. When possible, this is based on the oldest pending thread for the routed party; otherwise it falls back to the latest activity from the opposite party, then to recent PR activity or PR creation time. Format: <1m, Xm, Xh, or Xd.

Merge conflicts remain visible without overriding normal CI, discussion, and approval routing. A conflicted PR can therefore stay with reviewers or maintainers when they can resolve it themselves. If missing required checks or a missing Copilot review hold the PR with its author, the live status names the conflict and the standard seven-day author reminder still applies. Missing automation does not expire into a dashboard delivery failure while the PR is conflicted.

How to opt in

Open a pull request that adds your repository to .github/scripts/pull-request-dashboard/repositories.json:

[
  {
    "name": "example-repo",
    "approver_teams": ["example-approvers"],
    "required_approvals": 1,
    "non_blocking_check_patterns": [
      "markdown-link-check / link-check",
      "codecov/*"
    ],
    "require_clean_copilot_review_branches": ["main"],
    "labels_to_display": ["size/*", "breaking change"],
    "slack_channel": "#example-maintainers",
    "slack_user_mapping": {
      "octocat": "U0123456789"
    }
  }
]

Fields:

Field Required Description
name yes Name of the repository under open-telemetry.
approver_teams yes GitHub team slugs whose members count as approvers.
required_approvals no Number of active approvals required for an open PR to be marked ready to merge. An approval stops counting while that reviewer has a pending re-review request. Defaults to 1.
labels_to_display no Case-sensitive shell-style label name patterns to display inline after PR titles. Exact names such as breaking change and wildcard patterns such as size/* are supported. Defaults to [], which displays no labels.
non_blocking_check_patterns no Check-name globs for non-required checks whose failures should be identified in the live PR status comment. When the PR is waiting on the author, matching failures are reported only when at least one required check is failing and are noted alongside those failures. On other routes, matching failures are shown separately. Matching checks remain informational and do not affect routing or the dashboard CI column.
require_clean_copilot_review_branches no List of base branch names for which a Copilot review of the current head with no open Copilot review threads is required before automatically routing a PR to reviewers or maintainers. An effective /dashboard route:reviewers break-glass override bypasses this gate and every other routing blocker for the current head. A thread counts as open while it is unresolved and GitHub has not marked it outdated, so a thread whose code the author has since rewritten stops holding the PR even if nobody resolved it. The live PR status links every open Copilot thread until it is resolved or outdated, including threads where the author has already replied. The dashboard re-requests Copilot review when a push has left the previous review stale, and requests the first review itself if automatic Copilot code review has not produced one within an hour of the PR becoming ready. It does not duplicate a pending request. A request counts as delivered only once GitHub confirms Copilot is a pending reviewer or finds a completed Copilot review of the current head, so one that GitHub accepts but does not record in either form is sent again on the next pass. A gate that never reports holds the PR for at most four hours; after that the PR routes anyway, its status comment says which gate the dashboard stopped waiting for, and the run reports the stall. A conflict resets this clock because GitHub may not start checks or reviews until the conflict is resolved. List only branches where automatic Copilot code review is enabled (typically ["main"]); PRs targeting any other branch are never gated, so they cannot stall waiting for a review that never runs. Defaults to [] (no branches gated).
slack_channel no Slack channel for notifications. Omit to skip Slack processing for this repository.
slack_user_mapping no Map of GitHub login to Slack user ID for at-mentions.
large_repo no If true, apply rendering presets that keep the dashboard body under GitHub's 65,536-character issue-body limit: cap each section (each Waiting on … table and the Draft pull requests table) at 100 rows, and omit the Draft pull requests section entirely. Truncated sections get a _More X PRs not shown_ footer. Defaults to false (no cap, drafts shown). Enable this for very large repos with hundreds of PRs.

labels_to_display only controls which labels are shown. It does not filter pull requests or affect dashboard routing, notifications, or status comments. All matching labels are displayed in the order returned by GitHub; a label matching more than one configured pattern is shown once.

For each listed branch, require_clean_copilot_review_branches expects automatic Copilot code review to produce the initial review, so list only branches where automatic Copilot code review is enabled. When no automatic review has arrived within an hour of the PR becoming ready, the dashboard requests one itself rather than holding the PR on its author indefinitely. The dashboard requests reviews using its GitHub App installation token with pull-request write permission. Leave Review new pushes disabled if the dashboard should request re-reviews only when a PR is ready to return to reviewers or maintainers.

Ask a maintainer or admin to add the repository under Repository access.

Once the PR is merged, the dashboard will pick up your repository on its next hourly backfill run. To run it sooner, see Manual backfill run. The dashboard issue is discovered dynamically in your repository by the dashboard label and Pull Request Dashboard title; if it does not exist, the publish step creates the label and issue.

Review feedback lifecycle

Inline review threads and top-level feedback have different lifecycles on GitHub. Top-level feedback means a standalone PR comment or submitted review summary that is not attached to an inline review thread.

  • An inline thread remains on the dashboard until someone marks the conversation resolved on GitHub or GitHub marks its code anchor outdated. A completed author reply hands the action away from the author but does not close the thread. If the PR has enough approvals, it resumes approval-based routing and can move to maintainers without reviewer confirmation; otherwise it waits on reviewers. A reply that commits to more work keeps the PR with the author. Author-only inline threads are treated as annotations rather than review feedback unless a non-author joins them.
  • Top-level feedback has no resolved state. The dashboard therefore tracks each actionable top-level feedback item independently. 📌 means that one or more of those items are waiting on the author.
  • Reviewer badges reflect GitHub's latest review state. A CHANGES_REQUESTED state affects only the reviewer's badge; it does not affect dashboard classification or routing. 🔴 remains until a later approval or dismissal clears that state. Empty review summaries are ignored; any inline comments are tracked through their own threads.
  • When a human reviewer is requested again, ⏳ replaces their previous approval badge and that approval stops counting toward required_approvals. The PR waits on reviewers unless its other active approvals meet the threshold. This is intentionally stricter than GitHub, which can continue to accept the old approval when the repository does not dismiss stale approvals. 🔴 stays alongside ⏳ while the changes-requested review blocks the merge. A later approval or dismissal clears 🔴, but ⏳ remains until the requested re-review arrives.

Closing top-level feedback

An explicit author reply is the only thing that closes an actionable top-level feedback item: a later standalone PR comment by the author that answers, decides, clarifies, or reports what was done. The reply lets authors explain why a suggestion was not applied, ask a clarifying question, or otherwise close the dashboard action. If the author instead commits to future work in the current PR, such as testing or making another change later, the item remains waiting on the author. The dashboard intentionally treats a reply as a handoff signal, not proof that the reviewer agrees with the outcome.

Routing after a reply

Item Before an author reply After an author reply When it clears
Inline author action Waiting on the author A completed reply resumes approval-based routing; a commitment to more work remains with the author The conversation is resolved or its code anchor becomes outdated
Top-level author action Waiting on the author; 📌 is visible Addressed; 📌 disappears and normal approval-based routing resumes Immediately after an explicit author reply

GitHub remains responsible for enforcing blocking review states when a maintainer attempts to merge.

This can route an approved PR to maintainers before a reviewer confirms the author's response. That handoff is deliberate: the open-thread badge remains visible so maintainers can merge or defer to the reviewer, while the dashboard avoids leaving an author marked as blocked after completing their part.

Live PR status comment

After the first full dashboard run has populated repository state, each targeted PR update creates or updates one dashboard-managed status comment on that PR. The comment presents the time its status was last refreshed in UTC and a compact Waiting on and Next step summary. The timestamp is refreshed on every targeted PR update, even when the dashboard status is unchanged. When the author has the next action, pending review feedback is summarized as a count and its inline-thread and top-level-feedback links are nested under that next step, followed by guidance for giving each item a clear outcome. When failing required checks and feedback both need author action, the two are listed under Next steps. At most 20 feedback links are shown across both groups; when more exist, a note reports how many of the total are shown. Draft PRs show that they are waiting for the author to move the PR out of draft to request review.

A failing required status check routes a human-authored PR to the author ahead of review and approval state. The live comment calls out required CI failures explicitly and combines that reason with review feedback when both need author action. When a repository configures non_blocking_check_patterns, matching failed checks are named in a note alongside the required-check action when the PR is waiting on the author because at least one required check is failing. On other routes, matching failures are shown separately. Optional check failures do not affect routing. When the base branch ruleset requires code scanning results, the check GitHub publishes for each configured tool is treated as required, including its neutral "could not determine the alerts introduced by this pull request" outcome, which holds the merge even though GitHub does not mark that check as required. PRs from unattended automation do not route to their author because the automation cannot act on a dashboard request. A Copilot-delegated PR can route to a human author recovered from an assignee or the first commit's committer. Known maintenance bots still route to maintainers after one approval.

A hidden marker lets the workflow update the comment in place and upgrade existing one-time guidance comments rather than creating duplicates. Status comments are refreshed automatically when their dashboard status or format changes, including on inactive pull requests. Clear outcomes keep stale or ambiguous feedback from being routed to the wrong person.

Reviewers should prefer inline comments for feedback requiring explicit resolution. See RATIONALE.md for the tradeoffs behind this behavior.

Targeted updates received before the first full dashboard run are ignored.

Reviewer routing override

When the dashboard says a pull request is waiting on its author, the author can comment /dashboard route:reviewers to request human help. The command is a break-glass handoff. It forces Waiting on reviewers regardless of review feedback, approvals, required checks, Copilot review, or merge conflicts. It is useful when the author is stuck or needs a reviewer or maintainer to explain what to do next. Members of the repository's approver_teams can use the same command.

The dashboard binds an authorized command to the head it sees when it first reads that command, and records that head in its acknowledgement reply. The handoff stays active while the pull request head matches the recorded one, so a later push restores normal routing and gates. Actionable human reviewer feedback posted after the command timestamp also ends the handoff, so normal routing and gates apply again. Praise, informational comments, bot feedback, classification failures, and feedback posted at or before the command timestamp do not end it. A push between the command and the pass that reads it is part of the same handoff, because the dashboard binds to the head it can actually see.

The dashboard replies to an unauthorized command explaining that only the author or an approver can use it, and replies to any unrecognized /dashboard command.

Author reminder

The dashboard posts one reminder when a pull request remains in Waiting on authors for one week. The friendly reminder @-mentions the author, links to the dashboard-managed status comment containing the current blockers, says that answering the open items resumes normal approval-based routing, and advertises /dashboard route:reviewers as a break-glass handoff for any other reason, including the dashboard routing the pull request wrongly. The same one-week clock applies while the author route is held on required checks, Copilot review, or merge conflicts.

When the dashboard routes the pull request to approvers or maintainers, it appends an italic note saying that the pull request is no longer waiting on the author, then marks the comment Outdated so GitHub collapses it. The original reminder remains available when the comment is expanded. If a temporary gate hold, episode reset, or removal from the dashboard ends the author-waiting episode, the appended note says only that the reminder no longer reflects the current dashboard state. Both notes link to the live status without naming the next route, which may change again. Leaving Waiting on authors also resets the one-week clock. If the pull request later returns to Waiting on authors and remains there for another week, the dashboard posts another reminder. Reminders are delivered by hourly runs when the pull request is next refreshed, so a due reminder in a large repository may wait for a later round-robin run.

Configuration

The dashboard separates calculation from delivery. The update job uses repository-scoped, read-only GitHub App access to calculate routing and persist pending work. A serialized publishing job holds pull request and issue write access and durably delivers status comments, author reminders, Copilot re-review requests, Slack notifications, and the dashboard issue. Successful deliveries are recorded on the state branch before another publishing run can start.

Each repository can route Slack notifications to its own slack_channel and map GitHub logins to Slack user IDs via slack_user_mapping. Repositories without slack_channel configured do not send Slack notifications.

Staged rollout

Dashboard changes reach repositories in two stages, so a regression shows up on a small set of repositories before it reaches everyone.

  • Canary repositories run the workflow and its scripts from the triggering commit and pick up a change as soon as it merges. The canary set is the CANARY_REPOSITORIES list in pull-request-dashboard.yml, which currently holds shared-workflows itself and opentelemetry-java-instrumentation.
  • Every other repository runs them from the promoted rollout ref: a release's commit SHA, hash pinned with the release tag as a comment.

The webhook queue follows the same boundary automatically. A merge deploys queueing only for the canary repositories. Merging the later promotion pull request triggers another deployment that enables queueing for every repository. No queue-mode variable or manual Netlify deployment is required.

Repository configuration is never staged. repositories.json is always read from the commit that triggered the run, so opting a repository in, or changing its settings, takes effect on the next run in both channels.

Promoting a change

  1. Merge the change. Canary repositories pick it up on their next run, and the webhook deployment automatically enables queueing only for those repositories.
  2. Let it soak.
  3. Cut a release with the Release workflow.
  4. Run the Promote pull request dashboard workflow from main and enter the release tag. It accepts only a newer, published, non-prerelease version and opens a pull request pointing every stable job at that release's commit, in both the uses: ref and the matching code_ref input.
  5. Review and merge the generated pull request to promote the release. The merge automatically redeploys the webhook with queueing enabled for every repository.

The stable refs are deliberately excluded from Renovate because advancing them is the production rollout. test_rollout.py also fails if the workflow and script refs disagree. The promotion workflow uses the repo-specific shared-workflows OTELBOT App to push the branch with workflow-file permissions and open the pull request so its normal checks run.

Rolling back

Revert the promotion pull request to an earlier release commit. If the reverted change bumped no delivery version, the stable channel resumes normally. If it did, those repositories stop delivering rather than repeat reminders they can no longer see as sent, so recovery is to roll forward with a fix. Canary repositories are unaffected either way, since they never run pinned code.

Changing the workflow

Running two versions at once adds two rules for changes to this workflow:

  • A new input on pull-request-dashboard-repo.yml is passed by the canary jobs only, until the promotion that teaches the rollout ref about it.
  • repositories.json changes must be additive for one promotion cycle, because pinned code reads live configuration. Renames and removals land after the code that understands them has been promoted.

Prerequisites

The target repository GitHub App must be installed on your repository. Follow the repository-access step under How to opt in.

Manual backfill run

To manually run a dashboard backfill, open the Pull request dashboard workflow, choose Run workflow, and populate the repository field with the target repository name under open-telemetry, for example opentelemetry-java-instrumentation. Leave repository empty to backfill every configured repository. Each run processes at most 50 PRs per repository; run it again to continue through larger repositories.