Allow setting worker container images at the WorkerPool or Space level

Feature RequestsWorkersSpacesSep 17

These settings would override any default images at the Org level, allowing us to define the three different ECR repos we use for Commercial, GovCloud, and FedRAMP respectively.

Workaround: No
Problem

❌ Rejected7

Bronze Rice

•

Oct 9

I suppose there’s an alternative option that would work. If you added a bulk-update mode to GraphQL, it could update multiple stacks at a time, and I could update the images that way, and tell my Terraform to just ignore the state for runner images.

Jonah Kowall
Team•
Oct 9
Hey Phil, there's a way to do this at the worker pool level today. Set SPACELIFT_DEFAULT_RUNNER_IMAGE on each pool's launcher (or spec.pod.defaultRunnerImage on the WorkerPool CRD if you're on Kubernetes). You could give Commercial, GovCloud and FedRAMP their own image and rotate it in three places instead of across 1,000 stacks. Docs here.
The catch is that it's a fallback. It only kicks in when nothing else sets an image, so you'd need to remove runner_image from your stacks and any .spacelift/config.yml once, and leave the account default empty. After that, a rotation is a pool config change and a worker roll. No admin-stack apply, and no need for a bulk GraphQL mode or ignore_changes.
A real space-level override isn't on the roadmap yet. I'll keep this post open to track that part. Let me know if the pool approach covers you.
- Bronze Rice
  •
  Oct 9
  Oh this will totally do the trick, thank you!
  Jonah Kowall
  Team•
  Oct 9
  Awesome, happy Friday to you. Do you want me to leave this open anyway, or should we close it off?
  Bronze Rice
  •
  Oct 9
  Hmm, yeah, I think you can close this. This is exactly the thing I needed.

Bronze Rice

•

Sep 23

Sorry for the delay, your reply got caught up in my spam filter, now fixed…

So, almost all the ~1000 stacks are in a single monorepo. We already manage the Spacelift stacks with an admin stack that updates in bulk. This is what takes an hour to apply.

There aren’t too many other full-stack changes that take quite this long because we leverage label auto-attachments and space inheritance where possible.

Right now the 1-hour runtime is manageable because we have so many workers available, but when we get to the end of onboarding we’ll have to scale down to a smaller fleet.

Jonah Kowall

Team•

Sep 17

Thanks for the detail, Phil. An hour per rotation across ~1000 stacks every couple of weeks is a total PITA, and the Commercial / GovCloud / FedRAMP split makes sense.

Upfront: Spacelift doesn’t support setting the runner image at the Space or WorkerPool level today. It’s resolved from the account default or the per-stack setting.

There may be a lighter path than updating 1000 stacks:

Runtime config — .spacelift/config.yml can override the runner image per stack. If stacks share a smaller set of repos, that can cut the number of places you edit a lot. (See runtime configuration YAML reference.)
Code-driven bulk update — The Terraform provider (or spacectl) can drive the runner image from one source value, so a rotation is one IaC change that propagates on apply. If you already manage stacks as code, that may already get you close to a “change a few things” workflow.

To point you at the right approach:

Roughly how are those ~1000 stacks spread across repos?
How do you manage stacks today — Terraform provider, Blueprints/Templates, or UI?
When you rotate the image, is it hand-edited per stack in the UI, or scripted?

That’ll tell us whether bulk update already covers it, or whether a Space/WorkerPool-level setting is the right gap to close. If that’s the right one, we can open this feature request up for others to weigh in on.

Thanks!

Linked to Worker launcher should allow to set another default docker imageSep 17

Allow setting worker container images at the WorkerPool or Space level

Comments7