How to Create Awesome Repeatable Project Setups for AWS CDK

Are you excited about using AWS Cloud Development Kit (AWS CDK) to define infrastructure-as-code?

Do you think that the AWS CDK project initialization is a bit clunky and does not give you what you need to hit the ground running with a new AWS CDK project?

Do you use Typescript? (by choice or by request)

If most of these have the answer yes, you do want to continue reading this article!

(Cover photo by ThisisEngineering RAEng on Unsplash)

What we want to accomplish

The goal is to make it easy and quick to set up a useful and productive project setup for use with AWS CDK.
A developer should within a few minutes have a setup that includes:

• Typescript compiler configuration
• Linting tool set-up installed and configured
• Unit-testing tooling in place and configured
• Sample code structure to start with
• Git project initiated
• AWS CDK base modules installed
• Workflows in place for validation and release (Github)
• Workflow for release to package registry (npm)
• Pull request templates in place (Github)
• Simplified project configuration

The list above is only a subset of features that we will have in place, but we will not go into all of them.

The magic sauce to make this happen is...

Most of the features mentioned above we will get from the tool Projen.
We will use that as a foundation to enable a project set-up that is relevant to you and your organization.
You can also read about Projen in How to simplify project setup with Projen, and other articles at Tidy Cloud AWS.

We will build our Projen-based project type, which you can use as a starting point for your project types.

You should have a simple custom project type in place in about 30-60 minutes ready to use by others if you follow the steps outlined here.

I do recommend that you first skim through the article here to get a gist of what it is all about.
After that, you can go through the steps in more detail.

Why use a tool like Projen?

Chance is that you, like I used to, copied various files from an old project whenever a new project should be set up.
If a colleague or friend needed a starting point, you just pointed to a repository (possibly a template repository) to copy from.

However, once you had copied the files from another repository, you were on your own.
You had to understand all the details that needed to be changed and you had to update these yourself.

With the project set-up we do here, maintenance of the set-up is much easier.
You do not need to understand and manage all nitty-gritty details, if you do not want to.

In the same spirit as infrastructure-as-code encourages scripted infrastructure set-up & maintenance, Projen encourages scripted project set-up and maintenance, i.e. project-setup-as-code.

Now that we have our goals defined, let us get started!

The stuff that you will need to start with

There are three things that we will need to create a project set-up for developers to use.
These three things also happen to be what the developers that use the set-up will need as well.

• Node.js - You can download and install from tthe Node web site. I recommend using the current long-term release (version 14.x).
• yarn - One of the Typescript/Javascript package managers. Install this globally after installation Node.js, for example running npm install yarn --global. You can use npm instead of yarn. However, the default is yarn in Projen, so we stick to that for simplicity, for now.
• git client - Download from here: https://git-scm.com

In addition, you should also have a Github account.
We will have workflows enabled for Github defined in our project set-up, so an account there is highly desirable.

Finally, you will also need an account for an NPM registry, such as https://www.npmjs.com.
This is for actually publishing your project type so it is available to others.
Other registries, such as Github packages are also fine to use.

We will cover the publication in more detail when we get to that point, so do not worry now if you do not have any NPM registry account yet.

Create our project type baseline

I assume now that you at this stage have node.js, yarn, and git command-line client installed, and that you have a Github account you can log in to.

1. Open a command-line window and create a directory where you will store the project type definition. This will become a git repository. Let us name it awesome-awscdk-project:

mkdir awesome-awscdk-project
cd awesome-awscdk-project

1. Create a project using projen for creating project types We can use Projen itself to create a project for defining project types for Projen... In our project directory, use Projen to create a project with project type jsii.

npx projen new jsii

The command npx is part of the Node.js installation and it will run an NPM package without requiring that it has been installed first.
If we use the command new with Projen, we can create a project of a specified project type.
The project type jsii is a building block to build a translation layer between Typescript/Javascript data structures and some other languages.
Projen uses some metadata description features of JSII, hence we use that project type to create our Projen project type.

The result of executing this command may look a bit like this:

❯ ls -la
total 656
drwxr-xr-x   20 erikl  staff     640 Aug 30 10:50 .
drwxr-xr-x   59 erikl  staff    1888 Aug 30 10:49 ..
-r--------    1 erikl  staff    4118 Aug 30 10:50 .eslintrc.json
drwxr-xr-x   13 erikl  staff     416 Aug 30 10:50 .git
-r--------    1 erikl  staff    1084 Aug 30 10:50 .gitattributes
drwxr-xr-x    4 erikl  staff     128 Aug 30 10:50 .github
-r--------    1 erikl  staff     831 Aug 30 10:50 .gitignore
-r--------    1 erikl  staff     431 Aug 30 10:50 .mergify.yml
-rw-r--r--    1 erikl  staff     372 Aug 30 10:50 .npmignore
drwxr-xr-x    4 erikl  staff     128 Aug 30 10:50 .projen
-rw-r--r--    1 erikl  staff     892 Aug 30 10:50 .projenrc.js
-r--------    1 erikl  staff   11358 Aug 30 10:50 LICENSE
-rw-r--r--    1 erikl  staff      14 Aug 30 10:50 README.md
drwxr-xr-x  656 erikl  staff   20992 Aug 30 10:50 node_modules
-rw-r--r--    1 erikl  staff    2898 Aug 30 10:50 package.json
drwxr-xr-x    3 erikl  staff      96 Aug 30 10:50 src
drwxr-xr-x    3 erikl  staff      96 Aug 30 10:50 test
-r--------    1 erikl  staff     827 Aug 30 10:50 tsconfig.eslint.json
-r--------    1 erikl  staff     827 Aug 30 10:50 tsconfig.jest.json
-rw-r--r--    1 erikl  staff  278145 Aug 30 10:50 yarn.lock

Luckily, we do not have to bother with the details of most of the files here.
There are essentially four places we care about:

• The .projenrc.js file - our project config
• The src directory - Our custom project type code
• The test directory - Tests, of course
• The README.md file - to describe what we provide here

Initial updates of the .projenrc.js file

In any project type created with Projen, the .projenrc.js file is the central point of configuration.
Open this file and you will see something like this:

const { JsiiProject } = require('projen');
const project = new JsiiProject({
  author: 'Adam Author',
  authorAddress: 'adam.author@example.com',
  defaultReleaseBranch: 'main',
  name: 'awesome-awscdk-project',
  repositoryUrl: 'https://github.com/adamauthor/awesome-awscdk-project.git',

  // deps: [],                          /* Runtime dependencies of this module. */
  // description: undefined,            /* The description is just a string that helps people understand the purpose of the package. */
  // devDeps: [],                       /* Build dependencies for this module. */
  // packageName: undefined,            /* The "name" in package.json. */
  // projectType: ProjectType.UNKNOWN,  /* Which type of project this is (library/app). */
  // release: undefined,                /* Add release management to this project. */
});
project.synth();

If you need, update the author and authorAddress fields.
Also, update the repositoryUrl to the name you want to use in Github for the repository.
For example, if you want to store in a Github organization you belong to.

The general idea here is that in this file we create an object with the appropriate project type (JsiiProject in this case).
We do project configuration with the parameter fields we pass to this object.

Create a Github repository

At this point, login to Github and create an empty repository with the same name as specified in the repositoryUrl field.

Once this is done, execute the following commands to add the Github repository as the remote for your local project repository - adjust the actual repository URL as needed:

git remote add origin git@github.com:adamauthor/awesome-awscdk-project.git
git branch -M main
git push -u origin main

This should get the initial project set-up committed

Our initial project updates

Let us make some simple changes to get a feel for the workflow when working with Projen-based projects.

We can remove the commented out fields in our .projenrc.js file, and we can add a description field, describing our project.

I am a fan of the Jetbrains IDEs, so I tend to use these in many projects.
In this case, let us exclude the IDE config settings from our repository, by adding .idea/ to our .gitignore file.
However, we do this through .projenrc.js.
The updated file should look something like this:

const { JsiiProject } = require('projen');
const project = new JsiiProject({
  author: 'Adam Author',
  authorAddress: 'adam.author@example.com',
  defaultReleaseBranch: 'main',
  name: 'awesome-awscdk-project',
  repositoryUrl: 'https://github.com/adamauthor/awesome-awscdk-project.git',
  description: 'Collection of awesome project types for setting up AWS CDK-based apps with Typescript',
  gitignore: [
    '.idea/',
  ],
});
project.synth();

To get these updates reflected in the generated configuration files, we run the command

npx projen

This command will execute the .projenrc.js file and perform any updates that are described there.
If you check what files have changed in the project, you may see something like this:

So essentially, if we do any changes in .projenrc.js, we then run npx projen to generate the updated files.
Typically, we do not touch any configuration files directly, besides .projenrc.js.

Defining our project type

We are going to define a very simple custom project type, which we call awscdk-closedsource-app-ts.

When you create an AWS CDK App project with Projen, it sets several default entries, which for the most part are good.
However, if you are building a closed source solution, some defaults may not necessarily suit you.
Two examples of such defaults are:

• The project includes an Apache 2.0 license by default, which may not necessarily be what you want.
• A project includes a Mergify configuration. Mergify is a nice tool, but you may not have an account for it and it is not free for closed source projects.

So as an example of a new project type, we will define our new project type to be identical to the bundled awscdk-app-ts, except for the default settings of these two features above.

Let's make a unit test!

In the test directory, we can remove the sample test file hello.test.ts and create our test file.
Let us name that awscdk-app-closedsource-ts.test.ts.
Our project set-up includes the installation of the Jest test framework by default, so we can define a test using that.

Here we will assume that we will have a class AwsCdkClosedSourceTypeScriptApp, which represents our new project type.
We will import this from our src directory, create a project instance from it, and check settings in it, in this case for Mergify:

import { AwsCdkClosedSourceTypeScriptApp } from '../src';

describe('Check settings specific for project type', () => {
  test('Mergify is not enabled for closedSource default', () => {
    const project = new AwsCdkClosedSourceTypeScriptApp({
      cdkVersion: '1.116.0',
      defaultReleaseBranch: 'main',
      name: 'myproject',
    });

    const hasNoMergifyConfig = project.github && !project.github?.mergify;
    expect(hasNoMergifyConfig).toBeTruthy();
  });
});

The documentation for the classes of the bundled project types is at https://github.com/projen/projen/.
In this documentation, you can see that the property github includes a mergify entry, which will define if the Mergify configuration is used.
The default if the github entry is not specified, is that it will be included.
So in our test, we can check that this configuration is not in place, after creating a project with the mandatory parameters.

NOTE

The test above is not a very good test to be honest.
Consider it just a simple example for illustration purposes.
In the https://github.com/cloudgnosis/aws-cdk-app-templates repository there is an improved version of the unit tests, which does a better job of testing the project type.
A partial extract of the code looks like this:

import * as fs from 'fs-extra';
import { AwsCdkClosedSourceTypeScriptApp } from '../src';
// @ts-ignore
import { mkdtemp, synthSnapshot } from './testutil';


describe('Check settings specific for project type', () => {
  let tmpTestDir: string = '';
  const originalConsoleError = console.error;

  beforeEach(() => {
    tmpTestDir = mkdtemp();
    fs.ensureDirSync(tmpTestDir);
    console.error = jest.fn();
  });

  afterEach(() => {
    fs.removeSync(tmpTestDir);
    console.error = originalConsoleError;
  });

  test('Mergify is not enabled and no license file for closedSource default', () => {

    const project = new AwsCdkClosedSourceTypeScriptApp({
      outdir: tmpTestDir,
      cdkVersion: '1.116.0',
      defaultReleaseBranch: 'main',
      name: 'myproject',
    });

    const snap = synthSnapshot(project);
    expect(snap['.mergify.yml']).toBeUndefined();
    expect(snap.LICENSE).toBeUndefined();
  });

The test code in this case generates the output files in a temporary directory, and the result read from disk.
Tests can then be performed on the generated output, and tests are not dependent on what the project object exposes.

Let us make our project type!

Now, the next step is that create the source for our project type.
We do that with two changes:

1. In the sample index.ts file, we replace its content with:

export * from './awscdk-app-closedsource-ts';

1. We create a source file awscdk-app-closedsource-ts.ts with the following content:

import { AwsCdkTypeScriptApp, AwsCdkTypeScriptAppOptions } from 'projen';

export interface AwsCdkClosedSourceTypeScriptAppOptions extends AwsCdkTypeScriptAppOptions {
  /**
   * If set to true, some default values are modified compared to the settings for AwsCdkTypeScriptApp
   * Specifically, the following default values are changed:
   * - licensed is false by default
   * - githubOptions.mergify is false by default
   * @default The default is true.
   */
  readonly closedSource?: boolean;
}

/**
 * Closed source CDK App, Typescript
 *
 * @pjid awscdk-closedsource-app-ts
 */
export class AwsCdkClosedSourceTypeScriptApp extends AwsCdkTypeScriptApp {
  constructor(options: AwsCdkClosedSourceTypeScriptAppOptions) {
    super({
      licensed: options.closedSource === undefined ? false : !options.closedSource,
      githubOptions: {
        ...options.githubOptions,
        mergify: options.closedSource === undefined ? false : !options.closedSource,
      },
      ...options,
    });
  }
}

There are a few points to make about the code here:

• For our new project type, we inherit from the existing AwsCdkTypeScriptApp, so we get all its capabilities "for free".
• We add an optional convenience parameter called closedSource. Its default is what we expect it to be with this project type.
• We add comments describing our class and fields. Projen will automatically generate API documentation for it when we build and package the project type.
• We add the metadata entry @pjid in the comments to the project type class AwsCdkClosedSourceTypeScriptApp. This data entry is the exposed project type name on the command line.

This is all we have to do for our custom project type, in this example.
For your project types, you will likely do more. An example is to add other default files in src and test.
In this case, I recommend that you will have a look at the classes SampleFile, SampleDir, and Component in Projen, as a starting point.

With the current code in place, you will likely want to check that it works.
For this, you can run

npx projen test

If the code is ok, then the tests will pass.

Packaging and publishing our project type

Before we can get ready to package and publish our custom project type, we need a bit of additional configuration, plus we need an NPM account.

Additional configurations

There are a few items that you may want to add:

• Minimum node.js version. Projen will default to v10 for the workflows, which is a bit old. Github will complain in this case. I choose to set this to v14, specifically to a version that Github accepts here.
• Dependency to projen. I add projen as a peer dependency, the user of the project will need to have it anyway.
• NPM access setting. This is an optional setting, and it depends on whether you choose to publish a scoped package or not. A scoped package will by default have access set to PRIVATE, which may not be what you want.
• keywords. You may want to add some keywords to facilitate a search for your project type after publishing it.
• scoped package. This is an optional setting and depends on whether you want to publish a scoped package or a global package. The example below includes a scoped package name.

The .projenrc.js file will look like this, with these additional configurations:

const { JsiiProject, NpmAccess } = require('projen');
const project = new JsiiProject({
  author: 'Adam Author',
  authorAddress: 'adam.author@example.com',
  defaultReleaseBranch: 'main',
  name: '@example/awesome-awscdk-project',
  repositoryUrl: 'https://github.com/adamauthor/awesome-awscdk-project.git',
  description: 'Collection of awesome project types for setting up AWS CDK-based apps with Typescript',
  minNodeVersion: '14.15.0',
  npmAccess: NpmAccess.PUBLIC,
  peerDeps: [
    'projen',
  ],
  gitignore: [
    '.idea/',
  ],
  keywords: [
    'aws',
    'cdk',
    'projen',
    'typescript',
  ],
});
project.synth();

Run npx projen to perform the needed configuration file updates.

You can now also run

npx projen build

which will compile, test, generate docs and perform packaging steps locally.
If this all works fine, you should be good to go for the final steps before publishing your project-type package!

NPM account settings

To publish your package to the public NPM registry, you need an account there.
It is free to sign up and register an account at NPM, so go ahead and do that, if you do not have an account already.

An important consideration here now is if you want to be able to publish scoped packages or not.

A scoped package looks like @example/awesome-awscdk-project, rather than awesome-awscdk-project.
The latter name is global, the former is within the scope @example.

To publish a scoped package, you must have an organization in NPM.
It is free to create an organization, as long as you only publish public packages.
If you want to publish private scoped packages, you need to pay for this capability.
In your NPM account, you have an option to add an organization:

If you want to publish a scoped package, also make sure you update .projenrc.js.
Either you change the name field to be the name of the scoped package, or you add the field packageName and set that to the name of the scoped package.

The final piece of the puzzle is an NPM access token.
This is your way to let the automated Github workflow that will release and publish your project type access NPM.

In your NPM account, choose the Access tokens.
Click on Generate new token.

For the Github workflow, you want to use the Automation token type.

Copy the generated token, since this is the only time you will be able to get it.

Github access to NPM

1. In the Github web interface, go to your project type repository.
2. Select Settings, then Secrets.
3. Click on New repository secret.
4. Enter the name NPM_TOKEN, and paste the access token value from NPM here.
5. Click on Add secret

Release and publish

With all these settings done, you can commit all changes and perform a git push.
If you commit to the release branch defined in the project, main, the defined release workflow will execute automatically.
It will build and package a release of your project. It will also label it with a version number, starting from 0.0.1.

Any update on the main branch, or a pull request merged into the main branch, will generate a new release.
If you are on major version 0, this will be a new patch release, e.g. 0.0.1 => 0.0.2 => 0.0.3 etc.

The release workflow follows Conventional commits and Semantic versioning.
For example, if the major version is > 0 and a commit message on the release branch includes "feat: some message...", then it will automatically increase the minor version number.
Other messages will just increase the patch version number.
You set the major version explicitly in the .projenrc.js file, with the majorVersion field.

Using the custom project type

To use a custom project type with Projen, you simply add a --from option when you create a new project:

npx projen new --from @example/awesome-awscdk-project awscdk-closedsource-app-ts

If your package only has a single project type defined, it is possible to skip the project type name, it will be used anyway.
If you have multiple project types in the package, you have to specify its name.
If you do not specify the name in that case, it will list the available project types.

Final remarks

I hope this guide for creating your project types was useful to you!
The Projen documentation is not crystal clear on how to do this (yet).
It is still a version 0.x project, but quite useful anyway in my opinion.

You can see a published example at https://www.npmjs.com/package/@cloudgnosis/aws-cdk-app-templates, with the source code at https://github.com/cloudgnosis/aws-cdk-app-templates.

This package contains multiple types, including the type defined here in this article.

NOTE: While you can technically delete a published package in NPM, it is discouraged and the preferred route is to deprecate it.
As far as I know, you have to do this via the npm command-line.

To use the npm deprecate command, I had to set an access token in the ~/.npmrc file, as I was not able to use the npm login command for this.

//registry.npmjs.org/:_authToken=myaccesstokenhere

Happy coding!