Allow push policies to cancel tasks

You can end up with an infinitely delayed tracked run in the following situation:

  1. Have a task that ends up in an Undecided approval state and is thus waiting indefinitely for approval

  2. Have a tracked run triggered by any means (manually, push policy, etc.)

  3. The tracked run is blocked by the pending task. The tracked run sits in the Queued state until the task is manually approved or rejected.

We had a tracked run sit in queued for 4 days unintentionally because of a task that was pending approval.

While we can notify for tasks pending approval, I have a feeling that we would want tracked runs to take precedence over tasks (if this undesirable, then you could lock the stack perhaps). We could theoretically implement this ourselves using a push policy, except that in reality, task runs pending approval don’t appear in the push policy input in_progress_runs property.

Workaround
We'll probably add notifications for task runs pending approval as well as add a recurring audit job to check for jobs that have been queued for a long time.
Problem
We require task approvals for anyone outside our Cloud Operations team to protect against sensitive Terraform commands being run on production infrastructure.

Please authenticate to join the conversation.

Upvoters
Status

⬆️ Gathering votes

Board

💡 Feature Requests

Date

9 months ago

Subscribe to post

Get notified by email when there are changes.