Skip to main content

Execute Ansible-vendor tracked runs in stack concurrent

Feature Requests

As a platform engineer, we are building a platform (currently in proof-of-concept stage) that uses Ansible-vendored Spacelift stacks to execute Ansible playbooks. These stack runs are triggered by orchestration via API such that each stack run executes against one EC2 instance as they are launched. Due to the blocking nature of stack runs, this limits the stacks ability to run multiple Ansible playbooks concurrently for a stack. Tasks are not a valid alternative as they do not provide any benefit to the resource/host configuration management natively built into Spacelift stacks.

Jubran Nassar is familiar with our use-case.

If we end up selecting Spacelift as our tool for our platform, this will be important to solve, but marking as nice to have for now. We should know more in the next few weeks.

Status: 🔭 Discovery3 comments

Log in to comment and vote

Comments3

  • Green Rice

    •

    Apr 10

    Just following up here.

    We are currently in alpha stage of our platform and chose to use Spacelift as our centralized Ansible execution platform so getting this feature will be important for scalability. Feel free to reach out (including meetings) if I can help clarify this ask.

    I can’t seem to update the priority after submission.

  • Green Rice

    •

    Mar 9

    Sorry for the delay. When I mention “multiple Ansible playbooks”, it specifics outlines this situation:

    • Multiple EC2 instances for one application are launched at the same time (as part of a scale up event)

    • For each EC2 instance, orchestration is executed which includes sending an API call to SpaceLift to execute the Spacelift Ansible stack associated with that application. This means that all of the EC2 instance for this application map to the one stack in Spacelift. The Spacelift Ansible stack is ustom-executed such the Ansible playbook uses —limit in Ansible to only target the host being launched. For newly launched instances, we are not looking to apply the playbook to the entire inventory of hosts.

    • Once all the orchestrations submit their calls to the Spacelift stack, we can observe that each Ansible triggered stack run is executed sequentially, so the last one in the queue has to wait for the subsequent ones to complete.

    The impact of this is that launching our EC2 instances does not scale. For example, if an Ansible playbook took 10 minutes to run, then it may take 30 minutes for the 3rd-4th EC2 instance to receive its Ansible configuration as it waits for the other Ansible playbooks/stack runs to execute first.

  • Black Breeze

    •

    Mar 3

    @Shawn Hardwick would splitting a single stack with multiple eligible playbooks into multiple stacks with only one playbook per stack work for your use case?