CDK pipeline won’t restart after mutation

CDK Pipeline is a clever construct that makes continuously deploying your application and infrastructure super easy. The pipeline even has the ability to update itself or “mutate” if your commits include changes to the pipeline itself. This behavior is controlled by selfMutation property of the pipeline constructor and is true by default. Once the pipeline updates itself – it also restarts itself, so that new changes can take effect.

But if you create your CDK pipeline with regular AWS Pipeline as a base e.g.

const rawPipeline = new Pipeline(this, 'RawPipeline', {
   ...
 });
 
const pipeline = new CodePipeline(this, 'Pipeline', {
   codePipeline: rawPipeline,
   ...
});

suddenly the pipeline won’t auto-restart after the mutation. What is happening? Continue reading →

Share node_modules between CDK CodeBuildSteps

CDK makes it pretty easy and straightforward to create CodePipelines – define the pipeline, add build steps as needed, add deployment stages and you’re done. But imagine a scenario where you install Node dependencies in one step and then need to run some NPM scripts in another down the line. In order to avoid reinstalling the dependencies every time you can pass output of one step as an input of another:

const installStep = new CodeBuildStep("install-step", {
   input: sourceStep,
   commands: ["npm ci"],
   primaryOutputDirectory: ".",
});
 
const testStep = new CodeBuildStep("test-step", {
   input: installStep,
   commands: ["npm run test"],
   primaryOutputDirectory: ".",
});

Passing output of the installStep as input of the testStep copies everything created in the installStep – files, directory structure – including all dependencies installed into node_modules folder to the testStep, so any npm command in theory should work. But in reality they’d fail, telling you that some module or other is not found. The reason is that in addition to installing dependencies npm ci command also creates symlinks between some of them. Copying files from one step to another loses those symlinks. In order to rebuild them you need to run npm rebuild before running any npm commands in the consecutive steps. So the test step becomes

const testStep = new CodeBuildStep("test-step", {
   input: installStep,
   commands: ["npm rebuild", "npm run test"],
   primaryOutputDirectory: ".",
});

And this will throw no errors.

Easily verify yourself on Mastodon with GitHub

Mastodon unlike Twitter doesn’t have official (or paid, thanks Elon Musk) verification badge for account profile, but it does offer a way to verify yourself – by placing following link tag into HTML of a website/page you own (your blog for example):

<a rel="me" href="https://your.instance/@YourHandle">Mastodon</a>

where “your.instance” and “@YourHandle” are the Mastodon server you use and your profile handle respectfully. But what if you want to verify yourself with your GitHub profile? An ideal place would be something like a README.md file. Unfortunately every link placed into a markdown file automatically gets rel="nofollow", so it’s a no-go. But there is a way to do this. Continue reading →

Writing first Mastodon bot

Over the years I’ve written quite a few Twitter bots. But since Elon Musk took over – the bird site has become unbearable, so I, like many others, migrated to Mastodon. What Mastodon is, and how it operates is a whole another story, but for our intents and purposes it is similar to Twitter: there is a home timeline where posts from people you follow appear, and you can post to the timeline as well.

Back on Twitter I used to have a bot that would tweet one-liners from Pink Floyd lyrics every hour. Follow me along as I recreate it on Mastodon.

First and foremost you have to make sure the Mastodon instance you’re on allows bots. Some do, some don’t – read the server rules to find out. I am using botsin.space instance that is specifically meant to host bots. Continue reading →

Conditionally ignore terraform resource update

Let’s say you have following SSM parameter resource

resource aws_ssm_parameter private_key {
  name      = var.name
  type      = "SecureString"
  value     = var.key
  overwrite = true
  tags      = var.tags
}

The value of var.key variable changes every time terraform runs. But you need to be able to prevent value update based on some conditions (say, bool variable var.overwrite_old_value).

You can’t use overwrite = property, because if it’s set to false terraform will throw an exception attempting to overwrite the value.

You can’t use lifecycle { ignore_chanes = [...] } because it requires static attribute values and doesn’t accept variables, functions etc.

So how do you update the value only the condition is met? Continue reading →

Dynamic AWS provider in terraform

Recently I needed to create a backup vault resource in Cape Town region, but only if the region is enabled in the AWS account. Straight approach:

provider "aws" {
   region = "af-south-1"
   alias  = "af-south-1"
}

resource "aws_backup_vault" "af_south_1" {
   provider = aws.af-south-1
   name     = "default"
}

would throw exception if af-south-1 region is not enabled for the account. Terraform has the ability to create a resource only if certain condition is met (via count = meta property), but it cannot conditionally declare providers.

But we can conditionally redirect the provider. Continue reading →

Nested Loops in Terraform: Create a map from 2 lists

Recently I encountered a Terraform task in which I had a list of roles and a list of policies and I needed to create a AWS resource for every combination of role-policy. In a “regular” programming language this would be a simple nested loop. Thankfully Terraform 0.12 added for_each and for attributes to declare recurring resources. But two problems remained:

1. I needed some kind of way to nest these for declarations
2. for_each attributes requires a map with a unique key

So let’s tackle these problems one at a time. Let’s we have 2 lists:

locals {
   ROLES = ["developer", "analyst", "manager"]
   POLICIES = ["arn:1", "arn:2", "arn:3"]
}

Continue reading →

Dynamic LINQ to XML

Language Integrated Query (LINQ) is a cool feature of .NET languages like C# that allows you to perform SQL-like query right within the language against language’s data structures (lists, arrays etc.) But one drawback of LINQ – you have to know in advance, at compile time which fields to select, what filter conditions would be. Sometimes there’s a need to supply these at runtime – e.g. user selects which fields they want to see

Thankfully there exists Dynamic LINQ Library that allows you to supply LINQ parameters as a string akin Dynamic SQL. Here’s an example of such query from the library’s homepage:

var query = db.Customers
    .Where("City == @0 and Orders.Count >= @1", "London", 10)
    .OrderBy("CompanyName")
    .Select("new(CompanyName as Name, Phone)");

Now, one thing that LINQ can do is query XML. So in theory if we load, say, this XML:

<DATA_CENTER>
   <SERVER IP="1.2.3.4">
      <OS>Windows</OS>
   </SERVER>
   <SERVER IP="5.6.7.8">
      <OS>Linux</OS>
   </SERVER>
</DATA_CENTER>

into an XElement and run something like this

var query0 = myXElement.Elements()
          .AsQueryable()
          .Select("new (Attribute(\"IP\").Value as IP, Element(\"OS\").Value as OS)")

it would produce list of IPs and OSes. Unfortunately this doesn’t work. Continue reading →

Export Dynamic LINQ to CSV

LINQ allows to perform various queries against different data structures. Wouldn’t it be great if you could easily export result of a LINQ query to CSV? Fortunately you can! This article by Scott Hanselman explain how and culminates in cool in its simplicity code:

namespace FooFoo
{
    public static class LinqToCSV
    {
        public static string ToCsv<T>(this IEnumerable<T> items)
            where T : class
        {
            var csvBuilder = new StringBuilder();
            var properties = typeof(T).GetProperties();
            foreach (T item in items)
            {
                string line = string.Join(",",properties
                      .Select(p => p.GetValue(item, null)
                      .ToCsvValue()).ToArray());
                csvBuilder.AppendLine(line);
            }
            return csvBuilder.ToString();
        }
   
        private static string ToCsvValue<T>(this T item)
        {
            if(item == null) return "\"\"";
   
            if (item is string)
            {
                return string.Format("\"{0}\"", item
                      .ToString().Replace("\"", "\\\""));
            }
            double dummy;
            if (double.TryParse(item.ToString(), out dummy))
            {
                return string.Format("{0}", item);
            }
            return string.Format("\"{0}\"", item);
        }
    }
}

Continue reading →