Spacelift should only apply the changes in the planning rather than all changes if the stack run is not confirmed for a long time

Feature RequestsTerragruntSep 5, 2024

Here's one case we across:

Krush merge his branch that create a tracked stack run on April 26th (check the 1st screen shot)

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # module.firehose_streams["FireworkStagingCommerceEventToS3"].aws_kinesis_firehose_delivery_stream.firehose will be updated in-place
  ~ resource "aws_kinesis_firehose_delivery_stream" "firehose" {
        id             = "arn:aws:firehose:us-west-2:170553093583:deliverystream/FireworkStagingCommerceEventToS3"
        name           = "FireworkStagingCommerceEventToS3"
        tags           = {
            "Name"       = "FireworkStagingCommerceEventToS3"
            "fw:project" = "naboo"
            "fw:subteam" = "data_infra"
            "fw:team"    = "data"
        }
        # (5 unchanged attributes hidden)

      ~ extended_s3_configuration {
            # (7 unchanged attributes hidden)

          ~ processing_configuration {
                # (1 unchanged attribute hidden)

              ~ processors {
                    # (1 unchanged attribute hidden)

                  - parameters {
                      - parameter_name  = "BufferSizeInMBs" -> null
                      - parameter_value = "1" -> null
                    }

                    # (1 unchanged block hidden)
                }
            }

            # (1 unchanged block hidden)
        }

        # (1 unchanged block hidden)
    }

  # module.firehose_streams["FireworkStagingPublisherClientStatusUpdateToS3"].aws_kinesis_firehose_delivery_stream.firehose will be updated in-place
  ~ resource "aws_kinesis_firehose_delivery_stream" "firehose" {
        id             = "arn:aws:firehose:us-west-2:170553093583:deliverystream/FireworkStagingPublisherClientStatusUpdateToS3"
        name           = "FireworkStagingPublisherClientStatusUpdateToS3"
        tags           = {
            "Name"       = "FireworkStagingPublisherClientStatusUpdateToS3"
            "fw:project" = "naboo"
            "fw:subteam" = "data_infra"
            "fw:team"    = "data"
        }
        # (5 unchanged attributes hidden)

      ~ extended_s3_configuration {
            # (7 unchanged attributes hidden)

          ~ processing_configuration {
                # (1 unchanged attribute hidden)

              ~ processors {
                    # (1 unchanged attribute hidden)

                  - parameters {
                      - parameter_name  = "BufferSizeInMBs" -> null
                      - parameter_value = "1" -> null
                    }

                    # (1 unchanged block hidden)
                }
            }

            # (1 unchanged block hidden)
        }

        # (1 unchanged block hidden)
    }

  # module.firehose_streams["FireworkStagingPublisherClientUpdateToS3"].aws_kinesis_firehose_delivery_stream.firehose will be updated in-place
  ~ resource "aws_kinesis_firehose_delivery_stream" "firehose" {
        id             = "arn:aws:firehose:us-west-2:170553093583:deliverystream/FireworkStagingPublisherClientUpdateToS3"
        name           = "FireworkStagingPublisherClientUpdateToS3"
        tags           = {
            "Name"       = "FireworkStagingPublisherClientUpdateToS3"
            "fw:project" = "naboo"
            "fw:subteam" = "data_infra"
            "fw:team"    = "data"
        }
        # (5 unchanged attributes hidden)

      ~ extended_s3_configuration {
            # (7 unchanged attributes hidden)

          ~ processing_configuration {
                # (1 unchanged attribute hidden)

              ~ processors {
                    # (1 unchanged attribute hidden)

                  - parameters {
                      - parameter_name  = "BufferSizeInMBs" -> null
                      - parameter_value = "1" -> null
                    }

                    # (1 unchanged block hidden)
                }
            }

            # (1 unchanged block hidden)
        }

        # (1 unchanged block hidden)
    }

Plan: 0 to add, 3 to change, 0 to destroy.

Changes to Outputs:
  - naboo_redis_addr           = [
      - "firework-exq-staging.qerpgn.ng.0001.usw2.cache.amazonaws.com",
    ] -> null

You can see the planning output of the tracked stack run in 2nd screenshot, it show us there are only 3 changed resources
We don’t confirm the stack run, so it’s not deployed
We run terragrunt offline which may change the state file
I found the tracked run is not confirmed, so I confirmed it. (check 1st screenshot)
You can see the applying output of the tracked run in 3rd screenshot, it shows us there are 1 to add, 4 to change and 8 to destroy

      - weighted_routing_policy {
          - weight = 0 -> null
        }
    }

Plan: 1 to add, 4 to change, 8 to destroy.

What we expected is in a single tracked stack run, if we have 3 resources to change in planning step, there should be only 3 resources to change in applying step.

Workaround: No

❌ Rejected3

Jonah Kowall changed status to ❌ Rejected

Team•

Jun 23

Pinned

This feature request did not gather enough interest for us to move it forward at this time.

People can continue voting on it, and if interest increases, it can be reopened for consideration in the future.

Orange Pond

•

Jun 22

•

Merged request

•

2 votes

Terragrunt: allow opting into using standard plan artifacts

As per https://docs.spacelift.io/vendors/terragrunt/limitations#mocked-outputs-and-the-apply-phase spacelift specifically DOES NOT use the plan artifacts from the plan phase for TG runs.

The way that the documentation says it it kind of makes sense .. at least as a default ..

I wish there was a way to say "We know better, we'll make sure to avoid using mocked outputs in plans, etc etc" .. and have the regular TF functionality back :)

Like in our TG setup we only use mocks for "validate", but not for other commands ..

dependency "something" {
  config_path = "some path"
  mock_outputs_allowed_terraform_commands = ["validate"]
  mock_outputs = include.global.locals.mocks.some-module
}

So we’d be safe from this particular scenario that is being described.

The alternative seems mildly scary .. you may get entirely different stuff applied than was approved, especially if you manage your state outside of spacelift.

Salmon Jaguar

•

Oct 29, 2024

See https://docs.spacelift.io/vendors/terragrunt/limitations#mocked-outputs-and-the-apply-phase

The long and the short of it is that in TG mode, spacelift specifically does not use a plan artifact between the plan phase and the apply phase ..

The reasons why they do what they do make sense based on that doc .. BUT ..

I wish there was a way to say "We know better, we'll make sure to avoid using mocked outputs in plans, etc etc" .. and have the regular TF functionality back

Like in our TG setup we only use mocks for "validate", but not for other commands ..

dependency "something" {
  config_path                             = "some path"
  mock_outputs_allowed_terraform_commands = ["validate"]
  mock_outputs                            = include.global.locals.mocks.some-module
}

so that’s my 2c here .. I’ll also start a feature request specifically for this :)

Spacelift should only apply the changes in the planning rather than all changes if the stack run is not confirmed for a long time

Comments3