refactor: remove existing vpc endpoint configurations from base module and move into sub-module

[bryantbiggs]

Description

• remove existing vpc endpoint configurations from base module and move into sub-module
• BREAKING: VPC endpoint resources are removed in favor of sub-module and for_each creation of any VPC endpoints
• BREAKING: AWS provider has been updated from v2.70 to v3.15 for VPC endpoint changes and available attributes
• combine issue/example projects into one project

Motivation and Context

• due to VPC endpoint services being added all the time, and the ever growing complexity of the module, this change is to make the module more flexible and to allow users to choose what VPC endpoints they elect to use and attach to whichever resources they choose

Closes #454
Closes #462
Closes #560
Closes #582
Closes #603
Closes #604
Closes #607
Closes #608
Closes #611
Closes #613
Closes #618
Closes #626
Closes #630
Closes #631
Closes #633
Closes #634
Closes #638
Closes #640

Breaking Changes

Yes:

• BREAKING: VPC endpoint resources are removed in favor of sub-module and for_each creation of any VPC endpoints
• BREAKING: AWS provider has been updated from v2.70 to v3.15 for VPC endpoint changes and available attributes

How Has This Been Tested?

• I have tested and validated these changes using one or more of the provided examples/* projects
• I have updated and validated using the complete-vpc example

[antonbabenko]

Looks very good. Large cleanup. I commented just a few small things.

[antonbabenko]

Let's add create boolean flag to be able to disable this module the same way as we do in the VPC. For this reason there should be default in every variable.

[bryantbiggs]

I added the variable in 5c6a231 but its not working as I think it should. Its giving me an error as shown below, not sure why this isn't working, thoughts? (note - the complete-vpc example as its currently written does not work with the create functionality)

[antonbabenko]

This is one of my favorite error messages :) objects vs objects battle...

Try writing like:

endpoints = var.create ? tomap(var.endpoints) : tomap({})

Ref: https://github.com/terraform-aws-modules/terraform-aws-route53/blob/master/modules/zones/main.tf#L2

[bryantbiggs]

ah, perfect - thank you. I was able to resolve with 0416cec - check it out when you have some time

[antonbabenko]

Yes, this is the best way. :)

[bryantbiggs]

@antonbabenko friendly reminder 😉

[antonbabenko]

I just hit this issue when using Terraform 0.15.0 - hashicorp/terraform-provider-aws#8040 (comment)

Works well when I put a value there as a string:

region = "eu-west-1"

[antonbabenko]

I also verified all these terraform state commands. Works as expected.

[antonbabenko]

v3.0.0 has been just released. 🎉 🎉 🎉 🎉 🎉 🎉

[tabletcorry]

So if I am reading this correctly, the migration from 2.x to 3.x requires manual admin intervention to modify state for every use of this module, with the potential that a mis-typed state mv or state rm could damage production VPCs?

Or if we don't use state commands, we take a traffic double-blip on all endpoints as they are destroyed and then re-created.

This is a very disruptive breaking change. I hope the 2.x line will continue to get updates if they required.

[antonbabenko]

Version 2.x won't be getting updates. We support just one major release at a time.

The process of upgrading from 2.x to 3.x indeed requires manual runs of terraform state mv commands if you can't allow recreation of resources (most of the production loads do not allow this).

I am pretty confident that Upgrade Notes are correct, too.

[tabletcorry]

Ok, thank you for confirming.

I do want to express that this is quite painful for a large system where infrastructure-as-code has been implemented strictly, and virtually no one can run manual state commands. I realize there was some pain in maintaining this in the module, but the fix has pushed that pain onto every invocation of this module by every consumer.

[bryantbiggs]

hi @tabletcorry - we do our best to avoid breaking changes but as with all software projects, you reach a point where you have to make a significant change to continue making forward progress and best supporting the needs of those who use the module(s). the prior versions are all there for use and can be pinned (recommended) to avoid any breaking updates, but ultimately, it is up to each individual if they want to take on the challenge in updating in order to receive the latest changes. we do feel that this change was a long time coming - AWS releases VPC endpoints all the time and the prior solution was not scalable nor was it flexible enough for most folks (based on issues and PRs that we received). again, pinning the module version is going to be the safest route to avoid any disruptions until folks are ready to make the necessary changes to update (it is an opt in, its not something that is forced upon anyone)

[Routhinator]

How does ses_endpoint_security_group_ids get translated here? SES is missing from the "Complete" example and I don't see any way to attach security groups to the endpoints?

How do we map the state shifts?

[Routhinator]

So, not sure how we migrate individual security groups for endpoints still..

However for others:

SES Example:

    ses = {
      service        = "email-smtp"
      tags           = { Name = "ses-vpc-endpoint" }
      subnet_ids     = module.vpc.private_subnets
    },

I cannot figure out how to provide both module.vpc.private_subnets and module.vpc.database_subnets to subnet_ids though.. Every permutation of concat and flatten I could think of failed to make an array it would accept. In my case I need to provide the DB subnet access to the endpoint for backups.

[acdha]

In addition to ses / email-smtp, I ran into “efs” to “elasticfilesystem”. It seems like it might be useful to have a list of some of the more common ones since aws ec2 describe-vpc-endpoint-services --no-paginate --query "ServiceNames[*]" isn't super intuitive.

[bryantbiggs]

@Routhinator you cannot associate the VPC endpoint with more than one subnet in the same AZ and why I suspect your permutations are failing. if you are creating a VPCe for SES, its one association in each AZ (typically via the private subnets)

[github-actions]

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.