Concurrency Cancel Pending

[dclayton-godaddy]

Any previously pending job or workflow in the concurrency group will be canceled.

Any chance we can make this configurable? The expectation for concurrency is to ensure jobs are run in order and not concurrent. Cancelling jobs seems to defeat the purpose of ensuring order. I would envision a cancel-pending: true if we wanted this or at least allow us to set it to false if we don't want it.

https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#concurrency

[MichalAugustyn]

We have a concurrency set for an label/unlabel event. If 4 labels are added at once, two out of four actions are being cancelled.
I have tried to set cancel-in-progress: false but it does not help.

[david-simons]

Any previously pending job or workflow in the concurrency group will be canceled.

Also was expecting cancel-pending: false to do exactly that - but it does not :(

[MichalAugustyn]

Finally managed to make my workflow idempotent so I don't really care if it's triggered on every label event action anymore.

I know there might be cases where it would not be possible but I suggest to at least try. I didn't consider this as a solution before but it worked.

You can access all existing labels using ${{ github.event.issue.labels.*.name }} to get the list of labels.
I used that adding 4 new labels at once which triggered 4 actions but only 2 of them were ran.

I was worried that the value for this github.event.issue.labels might differ between runs but it seems its not the case. The latest triggered run always contains all 4 labels.

[bauergeorg]

It is possible to implement this option which @dclayton-godaddy explained? I need this feature, too!

[soundstep]

I think I'm having a similar problem.
I switched to this kind of condition:

    if: |
      contains(github.event.pull_request.labels.*.name, 'action-merge') &&
      github.event.pull_request.state == 'open'

But adding other labels as the end of a run:

        uses: actions-ecosystem/action-add-labels@v1
        with:
          labels: 'action-merge-error'

seems to cancel other jobs that were queued, as we are running only one at a time:

concurrency: 'release-plan-merge'

Is there any way to debug what's happening? I only see the job as "Cancelled":

[soundstep]

It seems that updating labels on action that uses pull_request.types: [labeled] updates jobs that are in the queue.
Still unsure how to debug Github events.

[yellowmegaman]

Please add cancel-pending: false
We need this badly, since we're running workflow to create new tags for software bundle, and it's really hard to implement without concurrent run limit for workflow, or at least repo.

[eropple]

I'm running into similar issues at a job concurrency level when trying to control how many runners are handling some platform-specific integration tests, so as not to rate-limit the API that they're talking to. A way to avoid canceling other pending jobs (and, ideally, a way to specify number of concurrent jobs--I can do two concurrent sets of tests without having things have problems) would really help out.

[Akeem7]

All

[SparrowBrain]

Yeah, just cancelling concurrent jobs screws up the idea of PR check. Pending and waiting should be the default not to talk about the option to configure it.

EDIT: nevermind, I see the intended purpose. I hope I can achieve what I want with if cancelled check

[chrispat]

Thank you for all of the feedback.

The current feature was geared for two specific scenarios. The first being the default behavior where you have a deployment to say production and you only want 1 deployment happening at any given time and you always want the latest code deployed.

The second scenario where you set cancel-in-progress: true is for making that PR checks are always running for the latest commit in the PR and as PRs are updated you want to make sure in progress jobs are killed so you don't waste resources.

We have had various discussions on the team around the other scenarios mentioned in the thread and we do hope to address them at some point in the future. However, at this time I do not have any concrete dates.

[widoz]

There might be cases where each workflow adds different informations and since the Actions are not only for Deployment in this scenario the pending canceled workflow will cause issues.

[AETylerCarrol]

@chrispat I am running into a similar issue; is there any update on whether this will be done?

[mmomtchev]

@chrispat I have an entirely different use-case: I have a number of jobs that could share caches (conan built packages), but currently if I run them in parallel, they will overwrite each other caches. I would like to have one job populate the cache and the others to use it. This is a useful feature that will save resources.

[andrewharp]

@chrispat I have the exact same usecase as @mmomtchev. I build a docker image on my EC2 instance that has multiple GHA runners, and obviously I only need to build the image once per run on each unique commit hash. However my options seem to be to run the build for every run even if redundant, or space out my runs by X# of seconds so they don't cancel themselves (or use one of the other workarounds found in this thread).

It was actually very surprising to find that it would cancel pending jobs even after setting cancel-in-progress to false.

[aaronsteers]

With respect, I'm disappointed by a lack of solution here. If a cancel-pending boolean is not desirable/tenable, we should at least have a max-queue-size option - whereby the default behavior now being 1 could be expanded to 3, 5, or 10. GitHub could enforce a limit on this to ensure queue size did not grow unboundedly, and it could be implemented in a backwards compatible approach with what we have now - since current behavior is essentially: max-queue-size=1.

@chrispat - I hope the team will consider this option.

[djthornton1212]

It would make sense to include some sort of concurrency option to queue a workflow/job/step (without cancelling it).

We used something similar in Jenkins. We could lock a resource like a function in our scripted pipeline. That way only 1 job could make use of that bit of code at a time. Worked great for potential race conditions.

[ben-z]

I needed this functionality so I made a quick-and-dirty spin lock mutex. It's working pretty well for me so far: https://github.com/ben-z/gh-action-mutex

A more involved explanation here: https://github.community/t/avoiding-conflicts-when-two-workflows-push-to-the-same-branch/233774/9

[JSantora]

+1 to above.

[bebosudo]

I'm currently using this action, but unfortunately it doesn't run jobs in the order they were queued, which is important when e.g. closing and reopening an issue/PR if external actions are being performed (installation and tear down of test environments).

[ben-z]

Oh hi, I must've missed the messages here. @santicapchase @JSantora this works at a job-level, a related issue here: ben-z/gh-action-mutex#12

@bebosudo I'm pretty sure the jobs are run in the order that they are queued. Here's the revelant code:
https://github.com/ben-z/gh-action-mutex/blob/4b87e07748cc12bfbfb478a065c717b4849302a0/rootfs/scripts/lock.sh#L22-L23
https://github.com/ben-z/gh-action-mutex/blob/4b87e07748cc12bfbfb478a065c717b4849302a0/rootfs/scripts/utils.sh#L68-L82
Perhaps in your scenario, the jobs enter the critial section out-of-order?

[dgillman]

@ben-z, this is awesome! Are you open to PRs? I'd like to add logic for the following:

• create a timeout for the wait loop
• delete the branch if the queue is empty

Let me know if you see any issues with those additions, and if you're willing to entertain the contributions.

Thanks!

[ben-z]

@dgillman PRs are welcome! There are existing issues for the two features you mentioned:

create a timeout for the wait loop

ben-z/gh-action-mutex#4

delete the branch if the queue is empty

ben-z/gh-action-mutex#16

I don't see any fundamental blockers for these. Please feel free to put up PRs!

[sergio-toro]

Yeah, this would be a great addition, we could use this to coordinate the deployments of the services based on merges to the repo.

But as the behavior now is to cancel pending jobs, we can't longer rely on files modified to know which services need to be triggered, forcing us to deploy all the services every time.

While a cancel-pending: false would help us release our monorepo services based on the Engineer's PR merge and the queue of the workflows becomes the release train.

Come on Github, it has been 2years since this was reported!

[dezren39]

cancel-pending: false would be great.

[Pikabanga]

Any updates on this? We also need this for an auto-merge job that has concurrency issues when several jobs try to merge at the same time. Having a lock allowing only one job at a time but not cancelling pending jobs would solve this.

[gustaff-weldon]

Lack of cancel-pending: false is a only blocker keeping us from using concurrency. We want all deployment PRs to be queued, not cancelled.
They should go in order. Currently we rely on https://github.com/softprops/turnstyle but is not being maintained...

[hitsthings]

Another vote for cancel-pending: false here. I have func test jobs that rely on a separate shared server and cannot be run concurrently without stepping on each others' toes on that target server. But two different branches that cannot run func tests concurrently should still both run func tests.

[andyjhull]

Almost the 2 year anniversary. We need this feature too. Can someone please prioritize this? Something tells me it wouldn't take that much time to complete.

Either cancel-pending: false or it might be a better idea to configure the max # of pending runs to guard against a large number of jobs queued up accidentally. i.e. max-pending: x or pending-queue-size: x

[niall-shaw]

Perhaps we need to open a new ticket for this, since this ticket seems to be answered - maybe no one at github is paying attention to answered tickets?

[hitsthings]

If you open one, I'd appreciate a link here so I can follow it.

[niall-shaw]

Opened a new discussion here:
https://github.com/orgs/community/discussions/63136

[Finesse]

A similar "unanswered" discussion: https://github.com/orgs/community/discussions/41518

[matt-findlay]

Would be great if someone can prioritize this.

[harpener]

Hi guys, our use case is that we need a passive queue mechanism, two levels of concurrency instead of just one. Our workflow for status checks should cancel pending tasks from the same group 1 (pull request), but keep pending tasks from a different group 2 (repository). Essentially, the workflow should never run in parallel, but should always be queued if it comes from a different pull request. Not being able to specify more groups would be fine as long as we can at least disable cancelling pending tasks at all. This is what we had in Jenkins CI. I find it hard to believe that this is such a rare use case unsolved for years.

[pksukumar]

Hi,
This is needed for our usecase as well. We are using Github CI in our organisation for to plan/apply terraform templates.
We group the concurrency based on terraform working directory. We don't want to cancel the pending actions and want all the pending actions from the same directory to go sequentially.
It would be great to have this feature.

[acovaci]

Yet another 👍 from me, very similar problem: We need a proper mutex behaviour in order to ensure sequential runs. It's important so that we ensure consistency of the concurrent merges over time, but also so that we don't lose traceability of what merge caused what action to change status.

[Clint-Chester]

Unfortunately stumbled across this issue as well, so created a simple GitHub Action to rerun cancelled jobs - https://github.com/marketplace/actions/rerun-workflows

Obviously there's plenty of enhancements that can be made to it 😄 but was simple enough to fix our problem. Feel free to open up a PR if there's enhancements you'd like to make.

[bcarranza]

ey github team , please, could you help us? we really need the feature
cancel-pending: false
because of every workflow is important to us, due we are working with testings and we need to make a lot of process before continue with the next one which is a new version of software that we need to apply later.
do you have some news about it?

[CannibalVox]

GitHub is nothing but pain but nothing can be done to stop execs from buying it. Someday I will work at a place that uses gitlab and I will be at peace.

[jfajardofrink]

as someone who just migrated from gitlab back to gha I can say with absolute confidence the grass is not greener

[runofthemillgeek]

@jfajardofrink care to elaborate? I miss GitLab and how it handled CI, the simplified/sensible YAML that was a pleasure to work with back in ~2019. I'm curious if things got worse in the last couple years.

[phalicz-apptio]

I also recently got surprised by a limit of pending jobs set to 1. I tested my solution with only 1 extra job running in parallel and did not expect that with 3+ I will get into trouble. There is no easy workaround for this limitation.

Additional setting would really help:
max-pending=4
or
pending-timeout-seconds=120

[3h4x]

1 currently running + 1 pending sucks.
Everything past 1 pending gets canceled even tho I set cancel-in-progress: false. Very illogical

[marekmlk]

👍 for cancel-pending: false