Infrastructure-as-code - where is the customer obsession, AWS?

AWS has a problem with its handling of customers’ adoption of infrastructure as code. This applies to CloudFormation, and by extension, to the AWS Cloud Development Kit (AWS CDK). This is part of a bigger problem with AWS, I think.

I am a fairly happy user of AWS CDK. It is a very nice tool in the toolbox for infrastructure-as-code. However, it also suffers from some drawbacks coming from CloudFormation, some of which will be mentioned here.

Meeting customers where they are

AWS prides themselves with customer obsession, and meeting customers where they are. They have a Well-Architected Framework which points to virtues of adaptability and automating things. Yet, for allowing customers to gradually move to a more automated and adaptable approach, there are sometimes too many hurdles in place.

It is, most times, fairly easy to use the AWS Console (ClickOps) to set up something with a particular AWS Service. There are also tutorials for using the AWS Console for many tasks.

Yet, there is zero support to get a corresponding representation in CloudFormation or AWS CDK from that ClickOps effort, so you can now manage those resources with infrastructure-as-code.

There is seldom any description or example to describe, in an easily digestible way, what you need to do to replicate that ClickOps experience.

I have yet to meet anyone using AWS that got a near-perfect infrastructure-as-code immutable infrastructure solution in place right away. There are always pain points and teachings on the way, and a process of adaption to new ways of working.

In addition, there is also the reality with operational incidents that need to be resolved quickly. This may most times involve some manual and direct updates, if you have not reached automation and immutable infrastructure nirvana yet. That is a reality.## Missing pieces in the puzzleYou cannot easily import existing resources into CloudFormation, and for a long time, it was not possible at all. Nowadays, you can import some existing resources into CloudFormation stacks, but it is difficult. It is also not supported for all CloudFormation resources either. There is a list in the AWS documentation of what resources are supported.It would have been better to have a list of what resources are not supported. This import feature only guarantees that required properties are the same as the actual resource when imported into CloudFormation.

The matter becomes even worse with AWS CDK since the import works on the CloudFormation level, which AWS CDK, to a large extent, abstracts away. It is then also potentially more complicated to replicate these resources through AWS CDK code.

There are tools, not from AWS, like Former 2, which can ease the pain somewhat, but it can only do so much and you get something less nice from a code perspective.If you are happy to replace all the existing resources, then this is less of a problem. But in many cases, this is not a good option if you still have a somewhat long way to go to immutable infrastructure nirvana.

Migration issues

Even migrating from CloudFormation to AWs CDK has its challenges. AWS CDK has some nice support to encapsulate existing CloudFormation in the AWS CDK code, which works pretty nicely. Just getting the AWS CDK command-line tool in the workflow is a nice bonus compared to what tooling is provided by AWS for dealing with CloudFormation directly.

You can keep the resources as-is with this approach, without replacing them. However, it becomes more challenging after that. You cannot easily break out parts of CloudFormation and replace it with AWS CDK code, without re-creating the infrastructure.

Again, not an issue with immutable infrastructure nirvana, but definitely a problem for many others.

Other solutions to the problem

If you want to find better options for this type of problems, look at the competition. Terraform from Hashicorp and Pulumi both do a better job of co-existing, reconciling, migrating, and gradually adapting your infrastructure. In this situation, both Hashicorp and Pulumi are better at meeting customers where they are.## Final words

I sometimes wonder why these kinds of pain points with customers are not addressed properly by AWS. Perhaps there is less incentive to let customers keep existing resources, instead of spinning up new ones to replace the old ones, since that provides more money to AWS. I really hope that is not the case. Or AWS persons are too used to immutable infrastructure nirvana in their daily job, that they do not see the pain points for customers here who are in earlier stages of the infrastructure-as-code journey.What do you think? Are these pain points for you?