Over the years of using AWS, Ive invested a lot of effort into managing costs and maintaining a tidy account. However, I began creating new accounts and closing them once they were no longer needed. This approach has led to several challenges, including:
Complications in billing management
Inability to share credits
Challenges with quota management
Orphaned resources
A $1 charge for each account created
The need to create temporary emails and manage contact information
AWS Organization
AWS Organizations is a service from AWS designed to streamline the centralized management of accounts. It offers features such as account provisioning, centralized billing, access management, policy management, and enforcement of standards.
Organizational Unit
An Organizational Unit (OU) is a grouping of accounts that allows for the distribution of accounts based on specific contexts or needs. For example, you might create one OU for workload accounts (development, testing, production) and another for security or networking purposes.
This separation enables additional automation for the member accounts, such as bootstrapping or deploying infrastructure tailored to specific contexts.
AWS CDK and challenges
Onboarding a new organization using CDK initially appeared to be as straightforward as creating any other piece of infrastructure. However, as I began implementation, I encountered several gaps due to missing features in CloudFormation and the Dedicated Service API.
The AWS Organizations service requires trusted access, which is only available through the Organizations Service API.
Activating SSO with IAM Identity Center can only be done through a manual process in the console.
Creating accounts necessitates unique email addresses, which makes it frustrating to set up multiple Gmail or whatever accounts.
While setting up SES with Route 53 was relatively easy, the documentation was confusing and misleading.
Source Code
https://github.com/XaaXaaX/aws-cdk-organization-setup
Phase 1: Configuration
Since setting up an organization involves more details than an application, its important to consider the relevant configuration sections. The following ConfigType outlines the structure of the stack configuration.
export type Config = {
contextVariables: ContextVariables;
dns: DNSConfig;
org: OrgConfg;
sso: SSOConfig;
}
DNS Config
DNSConfig should account for either importing an external HostedZone or creating a new one. By separating the types, we can enhance type safety and maintain better control within the stack.
type ExternalZone = { isExternal: true; hostedZoneId: string; }
type InternalZone = { isExternal: false; domainName: string; }
export type DNSConfig = (ExternalZone | InternalZone) & {
mailExchangeDomainName: string;
};
Org Config
OrgConfig includes configuration attributes related to account creation and bootstrapping, as well as the activation of trusted services and parameter sharing.
export type OrgConfg = {
memebers: {
bootstrap?: boolean;
accounts: {
accountName: string;
}[];
};
trustedAWSServices?: string[];
crossAccountParametersSharing?: boolean;
};
SSO Config
SSOConfig features two distinct types: Ready and NotReady. This distinction allows for the creation of SSO and identity store groups, along with permission sets, after implementing the Click-Ops solution in the management account.
export type SSONotReady = { isReadyToDeploy: false; }
export type SSOReady = {
isReadyToDeploy: true;
ssoInstanceArn: string;
identityStoreId: string;
}
export type SSOConfig = SSONotReady | SSOReady;
HostedZone and Emailing
The management account can either own an existing HostedZone or create a new one for a domain name, and also set up the necessary components to receive emails. This is crucial for having unique email addresses for each account while still allowing them to be forwarded to a personal account.
The process for importing or creating the hosted zone is outlined below:
let hostedZone: IHostedZone;
if(dnsConfig.isExternal)
hostedZone = HostedZone.fromHostedZoneId(this, 'HostedZone', dnsConfig.hostedZoneId);
else {
hostedZone = new HostedZone(this, 'HostedZone', { zoneName: dnsConfig.domainName, comment: 'Managed by CDK' });
new StringParameter(this, 'HostedZoneId', {
parameterName: `/${this.ENV}/${this.CONTEXT}/${dnsConfig.domainName}/hostedzone/id`,
stringValue: hostedZone.hostedZoneId,
})
}
To enable email reception through the HostedZone, an MXRecord must be added. Note that the mailExchangeDomainName configuration can be either the same as the domain name (e.g., example.com) or a subdomain (e.g., mail.example.com).
new MxRecord(this, 'MXRecord', {
zone: hostedZone,
values: [{
hostName: `inbound-smtp.${this.REGION}.amazonaws.com`,
priority: 10,
}],
recordName: dnsConfig.mailExchangeDomainName,
deleteExisting: true,
});
To receive emails using SES, you need to extend its capabilities through ReceiptRuleSet actions, as SES does not natively provide a mailbox option. To integrate SES, you must create an EmailIdentity, which should be of the Domain type, accomplished by providing the HostedZone.
new EmailIdentity(this, 'Identity', {
identity: Identity.publicHostedZone(HOSTED_ZONE),
});
The ReceiveRuleSet allows you to establish rules for receiving emails and define actions to be taken. The actions specified in a rule will be executed in sequence. In this example, each incoming email will be stored in an S3 bucket, and a Lambda function will be triggered to process the stored content and forward it to other email servers.
const receiptRuleSet = new ReceiptRuleSet(this, 'MailReceivedRuleSet', {
dropSpam: false,
rules: [{
tlsPolicy: TlsPolicy.REQUIRE,
scanEnabled: false,
enabled: true,
recipients: [ HOSTED_ZONE.zoneName ],
actions: [
new S3({ bucket: deliveryBucket }),
new Lambda({ function: receivedMailLambda }),
],
}],
});
Deploying the stack outlined above will create all the necessary resources for a functional domain name with email reception. However, if you attempt to send an email from your address, you will receive a postmaster response indicating that the email sending has failed.
inbound-smtp.eu-west-1.amazonaws.com has generated this error :
Requested action not taken: mailbox unavailable💡 SES permits only one active ruleset at a time, so when you create a new ruleset, it will not be activated automatically.
To activate the ruleset, you can use AWS CDK custom resources with SDK calls, as shown below.
const rulesetActivationSDKCall: AwsSdkCall = {
service: 'SES',
action: 'setActiveReceiptRuleSet',
physicalResourceId: PhysicalResourceId.of('SesCustomResource'),
};
const setActiveReceiptRuleSetSdkCall: AwsSdkCall = {
...rulesetActivationSDKCall,
parameters: { RuleSetName: receiptRuleSet.receiptRuleSetName }
};
const deleteReceiptRuleSetSdkCall: AwsSdkCall = rulesetActivationSDKCall;
new AwsCustomResource(this, "setActiveReceiptRuleSetCustomResource", {
onCreate: setActiveReceiptRuleSetSdkCall,
onUpdate: setActiveReceiptRuleSetSdkCall,
onDelete: deleteReceiptRuleSetSdkCall,
logRetention: RetentionDays.ONE_WEEK,
policy: AwsCustomResourcePolicy.fromStatements([
new PolicyStatement({
sid: 'SesCustomResourceSetActiveReceiptRuleSet',
effect: Effect.ALLOW,
actions: [
'ses:SetActiveReceiptRuleSet',
'ses:DeleteReceiptRuleSet',
],
resources: ['*']
}),
]),
});
This solution lets to activate the created ruleset and the ses reception now works as expected.
💡 The provided example only trigger a lambda function that logs an ses event but you can implement your email forwarding if needed.
Organization and Accounts
The CDK for AWS Organizations only offers L1 constructs, but I find this approach simple and straightforward. In my opinion, adding L2 constructs might be unnecessary over-engineering, so Im fine with this CDK decision for now.
To create an Organization and an OU, the only parameters required are the FeatureSet of the Organization, which can be either CONSOLIDATED_BILLING or ALL.
const orga =new CfnOrganization(this, 'Organization', { featureSet: 'ALL' });
const orgUnit = new CfnOrganizationalUnit(this, 'OrganitationUnit', {
name: `workloads${tempSuffix}`,
parentId: orga.attrRootId
});
orgUnit.addDependency(orga);
The following snippets illustrate how to create accounts. In this example repository, the account list is provided through configuration, meaning the accounts parameter will be an array of objects in the format { accountName: string }. The created account IDs will be stored in the parameter store, although this may not be necessary for a personal organization setup.
ACCOUNTS.forEach((account: { accountName: string }) => {
const awsAccount =new CfnAccount(this, `${account.accountName}Account`, {
accountName: `${account.accountName}${tempSuffix}`,
email: `${account.accountName}${tempSuffix}@${DOMAIN_NAME}`,
parentIds: [orgUnit.attrId],
});
const param = new StringParameter(this, `${account.accountName}AccountIdParam`, {
stringValue: awsAccount.attrAccountId,
description: `Account ID for ${awsAccount.accountName}`,
parameterName: `/${this.ENV}/${this.CONTEXT}/${awsAccount.accountName}/account/id`,
})
});
To set up SSO using IAM Identity Center, it's essential to enable AWS Organization trusted access. Unfortunately, theres no way to activate this feature using CDK or CloudFormation, so we will once again rely on a Custom Resource. In this example, all services required for trusted access are specified through configuration (e.g., sso.amazonaws.com and servicequota.amazonaws.com).
trustedServices.forEach((service: string) => {
const identifier = service.replace('.', '');
const enable: AwsSdkCall = {
service: 'organizations',
action: 'enableAWSServiceAccess',
physicalResourceId: PhysicalResourceId.of(`OrgCustomResource${identifier}`),
parameters: { ServicePrincipal: service },
};
const disable: AwsSdkCall = {
service: 'organizations',
action: 'disableAWSServiceAccess',
physicalResourceId: PhysicalResourceId.of(`OrgCustomResource${identifier}`),
parameters: { ServicePrincipal: service },
};
new AwsCustomResource(this, `AWSServiceAccessActivation${identifier}CustomResource`, {
onCreate: enable,
onUpdate: enable,
onDelete: disable,
logRetention: RetentionDays.ONE_WEEK,
policy: AwsCustomResourcePolicy.fromStatements([
new PolicyStatement({
sid: 'OrgCustomResourceSetOrgAWSServiceActivation',
effect: Effect.ALLOW,
actions: [
'organizations:enableAWSServiceAccess',
'organizations:disableAWSServiceAccess',
],
resources: ['*']
}),
]),
});
})
SSO Setup
The provided example configuration uses a NotReady state by setting isReadyToDeploy = false, which prevents the CDK deploy from generating the SSO configuration, as the SSO instance has not yet been created. As mentioned earlier, this cannot be accomplished through automation or API calls; the only available API call for CreateInstance works solely for standalone accounts, not for Organization Management accounts.
Before changing the flag to true, you must go to the AWS Console in the management account, navigate to IAM Identity Center, and click the Enable button. After activation, youll need the SsoInstanceArn and IdentityStoreId, which should be set in the stack configuration file along with isReadyToDeploy=true.
Once these steps are completed, the CDK deploy will proceed to deploy the stack along with all associated groups and permission sets.
const group = new CfnGroup(this, id, {
displayName: `${id}`,
description: `${id} Group`,
identityStoreId,
});
const permissionSet = new CfnPermissionSet(this, `${id}PermissionSet`, {
name: `${id}@${ENV}`,
description: `${id}@${ENV}`,
instanceArn: ssoInstanceArn,
managedPolicies: managedPolicies,
inlinePolicy: undefined,
sessionDuration: Duration.hours(12).toIsoString(),
});
accounts.forEach((account) => {
new CfnAssignment(this, `${id}Assignment`, {
instanceArn: ssoInstanceArn,
permissionSetArn: permissionSet.attrPermissionSetArn,
principalId: group.attrGroupId,
principalType: 'GROUP',
targetId: account,
targetType: 'AWS_ACCOUNT',
});
})
In the CDK snippet above, a group is created in the Identity Store, and a permission set is established for the SSO Instance. This group is assigned to each of the accounts created earlier. The code illustrates the Group Construct, which is utilized as shown below.
// Org Accounts
const developmentAccount = StringParameter.fromStringParameterName(this, 'AccountSecurity', `/${this.ENV}/${this.CONTEXT}/security_b/account/id`).stringValue;
//Managed Policies
const adminManagedPolicy = ManagedPolicy.fromAwsManagedPolicyName('AdministratorAccess');
const poweredUserManagedPolicy = ManagedPolicy.fromAwsManagedPolicyName('PowerUserAccess');
const readonlyManagedPolicy = ManagedPolicy.fromAwsManagedPolicyName('ReadOnlyAccess');
new Group(this, 'Admin', {
contextVariables: this.CONTEXT_VARIABLES,
ssoInstanceArn: SSO_INSTANCE_ARN,
identityStoreId: IDENTITY_STORE_ID,
managedPolicies: [ adminManagedPolicy.managedPolicyArn ],
accounts: [ developmentAccount ]
});
new Group(this, 'PowerUser', {
contextVariables: this.CONTEXT_VARIABLES,
ssoInstanceArn: SSO_INSTANCE_ARN,
identityStoreId: IDENTITY_STORE_ID,
managedPolicies: [ poweredUserManagedPolicy.managedPolicyArn ],
accounts: [ developmentAccount ]
});
new Group(this, 'Developer', {
contextVariables: this.CONTEXT_VARIABLES,
ssoInstanceArn: SSO_INSTANCE_ARN,
identityStoreId: IDENTITY_STORE_ID,
managedPolicies: [ readonlyManagedPolicy.managedPolicyArn ],
accounts: [ developmentAccount ]
});
You can now create a user in IAM Identity Center and assign it to one or more groups. Next, navigate to the AWS Access Portal (accessible via the link from the Identity Center dashboard: https://d-123456788.awsapps.com/start), which will prompt you for login credentials.
Once logged in, the application page will allow you to select the appropriate group role and access it with the corresponding permissions.
Account Boostrap
It was an effective way to automate account creation and implement varying levels of security. However, an empty account requires several repetitive tasks before it can be considered usable. The example includes setting up GitHub OIDC and CDK Bootstrap to prepare the member accounts.
The bootstrap can be deactivated through configuration, and this will be verified in the organization stack as shown below.
if( BOOTSTRAP ) {
new Bootstrap(this, 'Bootstrap', {
contextVariables: props.contextVariables,
regions: [ this.REGION ],
organizationUnits: [ orgUnit ],
types: {
[BootstrapTypes.CDK]: { FileAssetsBucketKmsKeyId: 'AWS_MANAGED_KEY' },
[BootstrapTypes.GitHub]: { Owner: gitHubConfig.owner, Repo: '*' }
},
})
}
The bootstrap construct creates a set of CloudFormation StackSets to allow the management account to bootstrap the member accounts, which will be triggered when accounts are created or updated. Unfortunately, the CDK does not offer a straightforward way to use CDK stacks for StackSets. After researching online, I found many solutions to be overly complicated, so I opted to stick with the readily available YAML templates found across the web (even though I probably wont look at or modify them). I'm fine with this approach.
export enum BootstrapTypes {
GitHub = 'oidc-github.yml',
CDK = 'cdk-bootstrap-template.yml',
}
const { contextVariables: { stage: ENV, context: CONTEXT }, types: TYPES } = props;
const tags = Stack.of(this).tags.renderTags();
Object.keys(TYPES).forEach((value: string, index: number) => {
const typeIdentifier = value.replace('.yml', '').replace(/[^a-zA-Z]/g, '');
const cfnParams = Object.entries(TYPES[value as unknown as BootstrapTypes])
.map(([key, value]) => (
{ parameterKey: key, parameterValue: value } as CfnStackSet.ParameterProperty
));
new CfnStackSet(this, `BootstrapStackSet${typeIdentifier}`, {
permissionModel: "SERVICE_MANAGED",
stackSetName: `${CONTEXT}-bootstrap-${typeIdentifier}-${ENV}`,
description: `Account bootstrap StackSet ${typeIdentifier}`,
autoDeployment: { enabled: true, retainStacksOnAccountRemoval: false },
capabilities: ["CAPABILITY_NAMED_IAM"],
templateBody: readFileSync(join(process.cwd(), `/cdk/lib/orga/bootstrap/${value}`), 'utf8'),
parameters: cfnParams,
tags,
operationPreferences: { failureToleranceCount: 1, maxConcurrentCount: 1 },
stackInstancesGroup: [{
regions: props.regions,
deploymentTargets: {
organizationalUnitIds: props.organizationUnits.map((ou: { attrId: string }) => ou.attrId),
},
}],
});
});
Conslusion
For a long time, I had been trying to set up an organization, but since I couldn't find a working piece of code, I decided to dive into it myself. It was an exciting experience, tackling different challenges and solving them along the way. Kudos to AWS CDK for its flexibility!
Having an organization is a great way to experiment and easily tear things down afterward. When closing accounts, youll incur charges for 90 days after theyve been removed from the organization. However, after this 90-day period, the accounts will be permanently deleted. During this time, you can still recover the account, access it with limited permissions, and perform certain actions.
Top comments (0)