Skip to content

CI: remove the Documentation Team Review Requests workflow - #117019

Open
groeneai wants to merge 2 commits into
ClickHouse:masterfrom
groeneai:groeneai/docs-review-requests-no-cancel
Open

CI: remove the Documentation Team Review Requests workflow#117019
groeneai wants to merge 2 commits into
ClickHouse:masterfrom
groeneai:groeneai/docs-review-requests-no-cancel

Conversation

@groeneai

@groeneai groeneai commented Aug 29, 2026

Copy link
Copy Markdown
Collaborator

Changelog category (leave one):

  • CI Fix or Improvement (changelog entry is not required)

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

The Documentation Team Review Requests workflow is removed.

Description

@ Blargian asked for the workflow to be removed rather than fixed, so that documentation
review requests can be driven from an existing workflow instead of a separate one:
#117019 (comment)

This is an intentional removal, not a fix. Once it merges, no documentation review request is
made at all until a replacement lands. Today they are made unreliably: 42 of the workflow's
300 runs in the 18 hours to 2026-08-29T07:45Z ended cancelled, and #116943 merged with zero
review_requested events although the routing owed it integrations-ecosystem and docs.

Only the workflow is deleted. ci/jobs/scripts/team_review_requests.py and
GH.request_team_reviews are left in place: they hold the path-to-team routing and the
request submission, so a replacement invoked from an existing workflow can call them
unchanged.

This branch previously carried a concurrency fix for the same problem. That is reverted here,
together with the ci/tests/ guard and the digest entry, which existed only to protect the
deleted file.

#116210 is my open pull request correcting the checkout revision in this same workflow. It
becomes unnecessary if this one merges, and I have left its disposition alone.

…l is added

`Documentation Team Review Requests` starts one run per `pull_request_target`
event, but only one event per pull request arms real work: `opened` for an
internal pull request, `labeled` with `can be tested` for a fork. Every other
event produces a run whose two jobs both evaluate `if:` to false and conclude
`skipped`. GitHub delivers the arming event once and nothing re-sends it, so the
runs at one head are not interchangeable and a discarded arming run cannot be
recovered.

Two GitHub behaviours were discarding it. `cancel-in-progress: true` cancels a
run that is already executing, and the default `queue: single` cancels and
replaces a run that is merely pending. A pull request normally receives three to
six of these events within seconds of being opened, so both fired routinely: 36
of 300 runs in one 16.5 hour window ended `cancelled`, across four head
repositories.

The `${{ github.event_name }}` term in the group looks like the cause but is not.
It was added when the workflow had two triggers, `pull_request` for internal pull
requests and `pull_request_target` for forks, purely to stop
`cancel-in-progress: true` cancelling across the two paths. Consolidating both
onto `pull_request_target` left the term constant, which widened the blast radius
to the internal path as well, but the term was only ever a mitigation for the
cancellation and never a fix for it.

Measured loss: ClickHouse#116943, docs-only and merged, had its `opened` run 33207978258
cancelled one second later by a `pr-autogenerated-docs` label run that then
skipped. It merged with zero `review_requested` events although
`get_docs_teams_to_request` returns `integrations-ecosystem` and `docs` for its
changed files. The nine other docs pull requests in that window, whose arming
runs survived, all received their expected teams.

`cancel-in-progress: false` keeps a running arming run alive and `queue: max`
raises the pending limit from one to 100. The group is left alone, so a pull
request's runs still serialise and two of them never race the same
`requested_reviewers` call; letting duplicates run instead of cancelling them is
safe because the job submits only the teams that are not already requested.
Writing `cancel-in-progress: false` explicitly rather than deleting the line
keeps the coupling legible, since pairing `queue: max` with
`cancel-in-progress: true` is a documented workflow validation error.

The `ci/tests/` guard pins both settings. The `ci_tests` digest gains this
workflow because the job digests `./ci` plus four explicit extra files, not
`.github/`, so without that entry a change confined to this workflow would leave
the job a cache hit and the guard would never run on it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@groeneai groeneai added can be tested Allows running workflows for external contributors groeneai-origin-ci-own-pr PR origin: CI on a groeneai PR exposed a separate defect labels Aug 29, 2026
@groeneai

Copy link
Copy Markdown
Collaborator Author
Internal second-model review: adjudication log (click to expand)

Pre-publication review by an independent model (engine: codex; 0 findings), plus my own cold
review of the resulting code: 0 blockers, 0 majors, 4 nits, no fix round needed.

# Sev Finding Verdict Evidence / action
1 💡 The new digest path points at a file that does not exist on release branches 26.3, 26.6 and 26.7 (only 26.8 was cut after that workflow landed), and Utils.traverse_path asserts on a missing include path, so a cherry-pick of this commit onto those branches would abort digest computation and fail Config Workflow there AGREE, not fixed Unreachable as written: this targets master, and the entry and the workflow file always travel together into a future release cut. Recorded so nobody applies a must-backport label to it. Not made tolerant of a missing path: that would be a praktika change with no failure behind it, and a loud assert is the better direction
2 💡 Related: sits inside the template's comment block, so it does not render DROPPED Raised, then withdrawn on measurement: the decisive provenance (#116943, run 33207978258) is visible in the description, and the comment block is where the template itself asks for relationship lines. Of the 40 most recently merged pull requests, 6 use that placement and 14 the visible one
3 💡 test_a_running_run_is_not_cancelled asserts is not True rather than == False DROPPED Not a defect, recorded so it is not "tightened" later. An absent cancel-in-progress key means false, so this pins the semantics rather than the presence of a line, and the mutant that deletes the line staying green is the control for it
4 💡 35 guard lines for a 5-line change reads as inverted AGREE, not fixed The two arms have disjoint mutant sets, so neither is redundant: deleting queue or setting queue: single reddens only the queued-run test, cancel-in-progress: true reddens only the running-run test, and master's file verbatim reddens both. The line count is a docstring plus assert messages, not extra arms. Please do not ask for more arms

Severity: ❌ blocker / ⚠️ major / 💡 nit. DISAGREE verdicts carry recorded evidence and are
terminal per finding.

queue is the one thing here that cannot be validated before merge, so I checked it three
independent ways. It is in the GitHub workflow JSON schema as concurrency.queue,
enum: [single, max], default: single, on an object declaring additionalProperties: false,
with the note that queue: max plus cancel-in-progress: true is rejected. GitHub's own
concurrency reference documents it behind a version flag that resolves to all of github.com.
And it is in production on a structurally identical workflow: yt-dlp/yt-dlp's
label-handler.yml pairs pull_request_target: [labeled] with cancel-in-progress: false
and queue: max across 2541 completed runs, including two created in the same second that
both completed instead of one cancelling the other.

I re-measured the description's countable claims on a fresh clock before publishing, including
9 of the 10 docs pull requests in the window getting every team they were owed while #116943
got none, and 3 of those 9 sampled against the raw API row by row.

#116210 is the only other open pull request touching this file. It changes the two ref: lines
about twenty lines below and does not overlap, so the two can merge in either order.

Session id: cron:clickhouse-review-slot-8:20260829-060400

@clickhouse-gh clickhouse-gh Bot added the manual approve Manual approve required to run CI label Aug 29, 2026
@clickhouse-gh clickhouse-gh Bot closed this Aug 29, 2026
@clickhouse-gh clickhouse-gh Bot reopened this Aug 29, 2026
@groeneai

Copy link
Copy Markdown
Collaborator Author
Pre-PR validation gate (click to expand)
# Question Answer
a Deterministic repro? Yes, as a deterministic observation rather than a local run: gh api repos/ClickHouse/ClickHouse/actions/runs/33207978258 returns conclusion: cancelled for the opened run of #116943, and issues/116943/timeline returns 0 review_requested events. Running the repo's own get_docs_teams_to_request over that pull request's 2 changed files returns ['integrations-ecosystem', 'docs'], so both owed teams were lost. Positive control: the same selector on #116951 returns ['docs'] and its timeline shows ["docs"] requested, so the probe discriminates. GitHub's scheduler cannot be run locally; the guard half of the fix has a fully local repro (below).
b Root cause explained? One pull_request_target event per pull request arms real work (opened internally, labeled can be tested on a fork) and GitHub delivers it once; every other event yields an all-skipped run. cancel-in-progress: true kills the arming run while it is in flight, and the default queue: single cancels and replaces it while it is merely pending. The runs at one head are therefore not interchangeable, and nothing re-arms a discarded one.
c Fix matches root cause? Yes. It removes discarding as a mechanism (cancel-in-progress: false + queue: max) instead of re-arranging the group key so the groups happen not to collide. The ${{ github.event_name }} term in the group looks like the bug but is not: it was a mitigation for the cancellation, made constant when the two triggers were consolidated.
d Test intent preserved / new tests added? New guard ci/tests/test_team_review_requests_concurrency.py, 35 lines, 2 tests, pinning the two settings that decide whether a run can be discarded. No existing test changed or weakened. The group expression is deliberately not asserted: it is no longer load-bearing.
e Both directions demonstrated? Yes. With master's file verbatim both tests fail; as shipped both pass. Confirmed under the CI-exact interpreter (clickhouse/integration-tests-runner, Python 3.13.14, PyYAML 5.4.1) and locally. A 4-mutant ladder gives one disjoint red set each: delete queue and queue: single redden only the queued-run test, cancel-in-progress: true reddens only the running-run test, and deleting the cancel-in-progress line stays green as the negative control, proving the assertion pins the semantics and not the line's presence. The workflow file was restored and md5-verified after every arm.
f Fix is general across code paths? Yes. Both jobs (request-reviews-internal, request-reviews-fork) are covered by the one mechanism, as is repeated delivery of the same arming event. All 26 workflows carrying a concurrency: block were enumerated mechanically: this is the only one setting cancel-in-progress: true and the only one triggered by pull_request*, and there is no job-level concurrency: anywhere, so there is no sibling to fix.
g Fix generalizes across inputs? N/A for datatypes and type wrappers (no C++ and no SQL surface). The event matrix is the analogue and is covered: opened, labeled with any label, and repeated identical events. Burst depth is bounded at 100 pending runs; the largest burst measured is 5.
h Backward compatible? Yes. No setting, no serialization format, no SettingsChangesHistory.cpp entry, no user-visible behaviour. queue is a valid workflow-level concurrency key: it is in the GitHub workflow JSON schema alongside group and cancel-in-progress, and queue: max appears in GitHub's own workflow-syntax reference. Were it unsupported it would fail workflow validation loudly rather than silently.
i Invariants and contracts preserved? The invariant is that a run carrying a one-shot arming event is never discarded to make room for a run that cannot do its work. Serialisation is deliberately kept, so two runs still never race the same POST /pulls/{n}/requested_reviewers; that is safe because the job submits only the difference against the teams already requested and re-reads to verify. The pull_request_target trust properties (base-revision checkout, persist-credentials: false, read-only permissions) are untouched.

Extra validation beyond the table: ci/tests/ was run on pristine master first and then with the change, giving byte-identical failure sets and exactly +2 passed (1224 to 1226), so no existing test moved. yamllint is clean in the real clickhouse/style-test image. The ci_tests digest entry was proven load-bearing with a 4-cell matrix: with master's include_paths the digest is identical before and after mutating this workflow (630 files, blind), and with the new entry it changes (631 files), so without it the guard would never run on the change it guards. Regenerating the praktika workflows was measured to be a no-op for this change: generated output is identical with and without the ci/defs/ edit, and the generator leaves this hand-written workflow byte-identical.

Session id: cron:clickhouse-impl-slot-4:20260829-053500

@clickhouse-gh

clickhouse-gh Bot commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Workflow [PR], commit [7b396af]

Summary:


AI Review

Summary

This PR deletes the standalone Documentation Team Review Requests workflow and intentionally stops automatic documentation-team review requests until a replacement is wired into an existing workflow. Given the current head commit and the prior discussion, I did not find a broken caller, stale workflow reference, or PR-metadata problem that warrants a review finding.

Final Verdict

✅ No review findings in the current diff.

@clickhouse-gh clickhouse-gh Bot added the pr-ci label Aug 29, 2026
@groeneai

Copy link
Copy Markdown
Collaborator Author

cc @Blargian, could you review this? Documentation Team Review Requests receives several pull_request_target events per pull request but only one of them arms real work, so cancel-in-progress: true plus the default queue: single were destroying the arming run: #116943 merged with neither of the two teams it was owed.

@Blargian

Copy link
Copy Markdown
Member

@groeneai I think we just need a better solution overall. Please remove Documentation Team Review Requests workflow and I will find out how to do it without the need for a separate workflow.

Blargian, who authored the workflow and the mechanism behind it, asked for the
workflow to be removed so that documentation review requests can be driven from
an existing workflow instead of a separate one:
ClickHouse#117019 (comment)

This is an intentional removal rather than a fix. With the workflow gone no
documentation review request is made at all until that replacement lands. Today
they are made unreliably: 42 of the workflow's 300 runs in the 18 hours to
2026-08-29T07:45Z ended cancelled, and ClickHouse#116943 merged without either of the two
teams it was owed.

It supersedes the concurrency fix this branch previously carried, so the guard
under ci/tests and the digest entry pointing at the deleted file are reverted.
Both existed only to protect that file.

ci/jobs/scripts/team_review_requests.py and GH.request_team_reviews are left in
place. They hold the path-to-team routing and the request submission, so a
replacement invoked from an existing workflow can call them unchanged.
@groeneai groeneai changed the title CI: do not discard the documentation review request when another label is added CI: remove the Documentation Team Review Requests workflow Aug 29, 2026
@groeneai

Copy link
Copy Markdown
Collaborator Author

@Blargian done in 7b396af: the workflow is deleted, and the concurrency fix with its ci/tests guard and digest entry is reverted, so this pull request is now only the deletion.

I left ci/jobs/scripts/team_review_requests.py and GH.request_team_reviews in place, since a replacement driven from an existing workflow can call them unchanged. Say the word if you would rather those went too.

This stops documentation team requests entirely until your replacement lands. If you want cover in the meantime, the previous commit 1902e065 is the two-line concurrency change and I can restore it.

#116210 is my other open pull request, fixing the checkout revision in the same file. It is redundant if this merges, and I have not touched it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

can be tested Allows running workflows for external contributors groeneai-origin-ci-own-pr PR origin: CI on a groeneai PR exposed a separate defect manual approve Manual approve required to run CI pr-ci

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants