Category Archives: AWS

Just let TypeScript do its thing

It is a well known fact that compile-time errors during development are easier and cheaper to catch than runtime ones in production. Unfortunately people keep forgetting that simple fact. Let me tell you a little story about AWS Lambda hiccup, and how it could easily be avoided.

A developer wrote a simple Lambda function in TypeScript that would receive a parameter of the following type:

interface Notification {
    slack?: {
        notify: boolean
    };
    github?: {
        notify: boolean
    };
}

The logic is simple – if slack.notify is true – send slack notification. If github.notify is true – send github commit status notification. The function deployed without issues, and it does send slack notifications. There is one problem: often it would send the same slack notification 3 times in a row. Continue reading →

Scan standalone CDK constructs for misconfiguration

Vulnerability scanners like checkov have become a de facto standard for scanning IaC for possible misconfigurations. They can scan terraform configs, AWS CloudFormation templates, even – in a sense – CDK stacks, because they synthesize into CloudFormation templates that can be scanned.

In order to produce synthesized templates from stacks you have to have a full-blown CDK application. But what if you’re developing a standalone CDK construct as a library, to be used as a dependency in a CDK app? There is a way to scan it for vulnerabilities as well.
Continue reading →

Reverse values into variables in tmplr

tmplr is a versatile templating tool that lets you quickly create a repository from a template customized for your needs. By running a recipe it can substitute placeholder variables with your own values making the repository truly your own. For example it can update package.json with dynamic values from git context. It can do much more, put for the purpose of this post we will concentrate on literal values in code files.

Let’s say you have a JavaScript code that performs some AWS SDK/CDK actions, and it requires AWS account IDs – for dev and prod deployment. In a regular repo you would define config.js something like this:

const awsAccounts = {
   dev: '1234567890',
   prod: '0987654321'
}

But if we’re building a template repo – we don’t want to hardcode those values. Instead we could use tmplr placeholder variables:

const awsAccounts = {
   dev: '{{ tmplr.dev_account_id }}',
   prod: '{{ tmplr.prod_account_id }}'
}

Save this file as config.tmplr.js – it becomes our template. A user of your template then can create a tmplr recipe:

steps:
  - read: dev_account_id
    eval: '1234567890'
  - read: prod_account_id
    eval: '0987654321'
  - copy: config.tmplr.js
    to: config.js

then save this it into .tmplr.yml file into repo root, and run tmplr cli command – it will create config.js file from config.tmplr.js substituting variables for values from the recipe.

It works great, but there is a problem. Continue reading →

Synthesize only stacks you need in CDK

CDK documentation states that if you supply stack name(s) to CLI commands like cdk synth, cdk list or cdk deploy – it will synthesize only the stacks you requested. But in reality this is not the case, CDK will always synthesize all stacks – and it may lead to unintended consequences.

Let’s say you have following stack declarations in your lib/my-stacks.ts code:

import * as cdk from 'aws-cdk-lib';

export class Stack1 extends cdk.Stack {};
export class Stack2 extends cdk.Stack {};
export class AnotherStack extends cdk.Stack {};
export class YerAnotherStack extends cdk.Stack {};

And in your app’s entry point bin/my-stacks.ts you instantiate those stacks:

import * as cdk from 'aws-cdk-lib';
import * as stacks from '../lib/my-stacks'

const app = new cdk.App();

new stacks.Stack1(app, "Stack1");
new stacks.Stack2(app, "Stack2");
new stacks.AnotherStack(app, "AnotherStack");
new stacks.YerAnotherStack(app, "YetAnotherStack");

And then issue a CLI command to synthesize stacks, but you only want to synthesize “Stack1” and “AnotherStack”:

cdk synth Stack1 AnotherStack

The command output would have you believe that only 2 stacks were synthesized:

Successfully synthesized to ./cdk.out
Supply a stack id (AnotherStack, Stack1) to display its template.

Continue reading →

Wait for AWS region to become available

AWS CLI has a nifty useful command to enable/opt-in a region on your account – account enable-region e.g.

aws account enable-region --region-name af-south-1

There is a caveat though – this command only begins to enable the region. The process could take a while, but the command exits right away. But what if you need do something with the region when it becomes available? Say you’re running a script and you need to bootstrap the region when its enabled. You need some way to wait for the enablement to finish. Luckily there is another AWS CLI command that lets you check the status of the region – ec2 describe-regions, e.g. 

aws ec2 describe-regions --region-names af-south-1

One of the properties it return is whether the region is enabled/opted-in or not. Combining this with a little of bash magic – we can come up with a waiting routine:

aws account enable-region --region-name af-south-1
state=not-opted-in
until [ "$state" = "opted-in" ]
do
   echo Waiting for af-south-1...
   sleep 5
   state=$(aws ec2 describe-regions --region-names af-south-1 --query "Regions[0].OptInStatus" --output text)
done

Here on the Line 1 we execute command that initiates enabling region. And the rest is a loop: wait 5 seconds, and check the status of the region. Loop exits when the status becomes “opted-in”. At this point we know that the region has been enabled, and can proceed with using it.

CDK pipeline won’t restart after mutation.

CDK Pipeline is a clever construct that makes continuously deploying your application and infrastructure super easy. The pipeline even has the ability to update itself or “mutate” if your commits include changes to the pipeline itself. This behavior is controlled by selfMutation property of the pipeline constructor and is true by default. Once the pipeline updates itself – it also restarts itself, so that new changes can take effect.

But if you create your CDK pipeline with regular AWS Pipeline as a base e.g.

const rawPipeline = new Pipeline(this, 'RawPipeline', {
   ...
 });

const pipeline = new CodePipeline(this, 'Pipeline', {
   codePipeline: rawPipeline,
   ...
});

suddenly the pipeline won’t auto-restart after the mutation. What is happening? Continue reading →

Share node_modules between CDK CodeBuildSteps

CDK makes it pretty easy and straightforward to create CodePipelines – define the pipeline, add build steps as needed, add deployment stages and you’re done. But imagine a scenario where you install Node dependencies in one step and then need to run some NPM scripts in another down the line. In order to avoid reinstalling the dependencies every time you can pass output of one step as an input of another:

const installStep = new CodeBuildStep("install-step", {
   input: sourceStep,
   commands: ["npm ci"],
   primaryOutputDirectory: ".",
});

const testStep = new CodeBuildStep("test-step", {
   input: installStep,
   commands: ["npm run test"],
   primaryOutputDirectory: ".",
});

Passing output of the installStep as input of the testStep copies everything created in the installStep – files, directory structure – including all dependencies installed into node_modules folder to the testStep, so any npm command in theory should work. But in reality they’d fail, telling you that some module or other is not found. The reason is that in addition to installing dependencies npm ci command also creates symlinks between some of them. Copying files from one step to another loses those symlinks. In order to rebuild them you need to run npm rebuild before running any npm commands in the consecutive steps. So the test step becomes 

const testStep = new CodeBuildStep("test-step", {
   input: installStep,
   commands: ["npm rebuild", "npm run test"],
   primaryOutputDirectory: ".",
});

And this will throw no errors.

Conditionally ignore terraform resource update

Let’s say you have following SSM parameter resource

resource aws_ssm_parameter private_key {
  name      = var.name
  type      = "SecureString"
  value     = var.key
  overwrite = true
  tags      = var.tags
}

The value of var.key variable changes every time terraform runs. But you need to be able to prevent value update based on some conditions (say, bool variable var.overwrite_old_value).

You can’t use overwrite = property, because if it’s set to false terraform will throw an exception attempting to overwrite the value.

You can’t use lifecycle { ignore_chanes = [...] } because it requires static attribute values and doesn’t accept variables, functions etc.

So how do you update the value only the condition is met? Continue reading →

Dynamic AWS provider in terraform

Recently I needed to create a backup vault resource in Cape Town region, but only if the region is enabled in the AWS account. Straight approach:

provider "aws" {
   region = "af-south-1"
   alias  = "af-south-1"
}

resource "aws_backup_vault" "af_south_1" {
   provider = aws.af-south-1
   name     = "default"
}

would throw exception if af-south-1 region is not enabled for the account. Terraform has the ability to create a resource only if certain condition is met (via count = meta property), but it cannot conditionally declare providers.

But we can conditionally redirect the provider. Continue reading →

Nested Loops in Terraform: Create a map from 2 lists

Recently I encountered a Terraform task in which I had a list of roles and a list of policies and I needed to create a AWS resource for every combination of role-policy. In a “regular” programming language this would be a simple nested loop. Thankfully Terraform 0.12 added for_each and for attributes to declare recurring resources. But two problems remained:

1. I needed some kind of way to nest these for declarations
2. for_each attributes requires a map with a unique key

So let’s tackle these problems one at a time. Let’s we have 2 lists:

locals {
   ROLES = ["developer", "analyst", "manager"]
   POLICIES = ["arn:1", "arn:2", "arn:3"]
}

Continue reading →