Skip to main content

Infrastructure As Code

We use AWS SAM to write our infrastructure as code. Our SAM templates have been written in a consistent way across all our stacks, to make it easier for new services/stacks to follow a blueprint. This document outlines these best practices and coding guidelines to adhere to when writing our infra as code, as well as general CF tips.

Setup​

SAM​

It's very useful to have SAM installed so you can test and validate our stacks locally. Follow this link to install the AWS SAM CLI.

ServerlessIDE: VSCode plugin​

I highly recommend this VScode plugin to make working with Cloudformation less daunting. It provides auto-completion support, and amongst other features auto links the resources you're writing to the AWS cloudformation docs which is extremely helpful.

VSCode syntax highlighting​

VSCode does not pick up on some of the YAML syntax used in Cloudformation, and can result in your screen being full of incorrect errors and red highlighting as it.

To disable these errors, you need to let VSCode know these syntaxes are valid YAML tags in our case.

  • Go to the VSCODE settings page (CMD + ,)
  • Search custom tags
  • Click the Edit in settings.json Under the Yaml: Custom Tags section.
  • Copy the below snippet into the settings.json
  "yaml.customTags": [
    "!Or sequence",
    "!Not sequence",
    "!Equals sequence",
    "!FindInMap sequence",
    "!GetAtt",
    "!GetAZs",
    "!ImportValue",
    "!Join sequence",
    "!Ref",
    "!Select sequence",
    "!Split sequence",
    "!Sub",
    "!If sequence",
  ],

Naming Conventions​

Please apply the following naming conventions to maintain consistency across the codebase. CF = Cloudformation

CF Template Filenames: template.yaml || snake-case.template.yaml for nested stacks\ CF Resources: PascalCase\ CF Parameters: pPascalCase\ CF Conditions: cPascalCase\ CF Mappings: mPascalCase\ CF Outputs: oPascalCase

The above covers the different type of properties that can be found within a cloudformation template. We prefix the type (i.e c for a Condition definitions) so that it is clear when looking at a resource definition, what specific type it is referecing to.

CF Resources definitions​

Resources:
  PlatformEventBus:
    Type: AWS::Events::EventBus
    Properties:
      Name: !Sub ${AWS::StackName}-event-bus
      Tags:
        - Key: Environment
          Value: !Ref pEnvironment
        - Key: Project
          Value: !Ref AWS::StackName
        - Key: Role
          Value: event-bus
        - Key: Name
          Value: !Ref PlatformEventBus

!Sub is a intrinsic function that does string substitions. See here for a list of all supported CF Instrinsic functions.

AWS::StackName returns us the current stack name. See full list of supported Pseudo parameters references See here

  • The above snippet provisions an Event Bus resource. Each CF resource has a Type and Properties.
  • The Type on the resource also correlates to a documentation page in AWS with the same title. (The plugin should link you to that page when you hover over the Type in your VScode.) From there you can see the allowed properties on the resource.
  • The top name of PlatformEventBus is the logical ID of the resource and is used for managing the state of that resource in stack updates, as well used by cloudformation to generate a resource name.

Resource names:​

In the above example, there is always a name property on each resource. This key varies based on the resource - i.e. it could be FunctionName, PolicyName etc on the context. In the case of an EventBus resource - the name is madnatory.

Where it is not mandatory - we should not define it and let Cloudformation define these names. Cloudformation will set the name as {stackName}-{ogicalId}-uniqueId. I.e. a Lambda with a logical ID of AutoApplyMovements in the inventory-service-events stack might get an auto-generated name of inventory-service-events-AutoApplyMovements-PL6kpwwYU1HV

  • The benefits of using these unique ID, means we wont have to worry about naming conflicts when creating multiple stacks. It also better allows updates of resources that require replacements.
  • Also it allows us to retain some resources such as our S3 buckets if we need to delete certain stacks, and to spin them back up. If the names were static we would have name clashes if we were ever to spin up the stack again.

If we have to provide the name ourselves, then use the pattern of {stackName}-resource-name as per the above snippet (!Sub ${AWS::StackName}-event-bus). This ensures the name is unique per stack.

More info on Resource names:

Tags​

Instead we should use tagging for resource discoverability in the AWS console. Using tags will also let us track costs better, more info on this here.

Our tags can be expanded as needeed but at the moment, there's 4 tags which should be applied:

  • Environment: The environment for the stack, i.e. dev.
  • Role: The role of the resource, i.e. event-bus. This should be consistent for the resource type.
  • Project: The project the resource belongs to, i.e. platform-infrastructure.
  • Name: A friendly name for the resource, i.e. platform-infrastructure-event-bus.

With the above 4 tags, it gives us much more querying support in the console, to find resources. It will also give better support to see our AWS bills based on these tags.

CF Mappings & Parameters​

Parameters:
  pEnvironment:
    Type: String

Mappings:
  mEnvironments:
    dev:
      PrivateSubnetIds: ['subnet-07086fc8a77a74c25', 'subnet-0dd0e8f5825b2a0ae', 'subnet-073fb73a22d545d6d']
    uat:
      PrivateSubnetIds: ['subnet-02ee167df71369ac2', 'subnet-0e6cdb395b68c6aec', 'subnet-034b882dbbabfed8e']
    sandbox:
      PrivateSubnetIds: ['subnet-0f3960c3bc36736b4', 'subnet-0e094d627b4878422', 'subnet-099aeb599f8e63100']
    prod:
      PrivateSubnetIds: ['subnet-09891c35b72352cb4', 'subnet-0d1644cae30583df3', 'subnet-0fe1565f9ed928417']

Parameters are injected in, and allows us to dynamically set values, and can have different values injected in based on the environment. Mappings is a map object where we can specifcy values based on a key, i.e. based on the active environment we're in.

From the above snippet, we have a pEnvironment which is injected in from our github workflow, and a mEnvironments map object which has different PrivateSubnetIds based on an env.

  ApplicationLoadBalancer:
    Type: AWS::ElasticLoadBalancingV2::LoadBalancer
    Properties:
      Subnets: !FindInMap
        - mEnvironments
        - !Ref pEnvironment
        - PrivateSubnetIds
      Type: application
      SecurityGroups:
        - !GetAtt ApplicationLoadBalancerSG.GroupId

As per the above snippet, you can see we do a !FindInMap to set the Subnets property accordingly.

Maps should be used where we:

  • Are referring to a resource/entity not controlled via IC. If the resource is defined via IC we should not use mappings.
  • Setting resource values, i.e. instance size based on an env (as this does not refer to a resource but a value.)

CF Conditions​

Conditions:
  cIsProd: !Equals [!Ref pEnvironment, 'prod']
  S3EventDataCrawler:
    Type: AWS::Glue::Crawler
    Properties:
      DatabaseName: !Ref EventsDB
      Role: !GetAtt GlueCrawlerRoleForS3Read.Arn
      Schedule:
        ScheduleExpression: !If
          - cIsProd
          - cron(0 * * * ? *) ## run crawler hourly in prod
          - cron(0 8 * * ? *) ## run crawler daily once at 8am for non-prod envs

There are cases where you want to apply different values per environment, or only provision resources in a certain environment. This can be achieved using Conditions, such as in the example snippet above.

  S3EventDataCrawler:
    Type: AWS::Glue::Crawler
    Condition: cIsProd
    Properties:
      DatabaseName: !Ref EventsDB
      Role: !GetAtt GlueCrawlerRoleForS3Read.Arn
      Schedule: ....

If you need to conditionally create a resource in only certain environments, then you can set a Condition attribute directly against your resource which links to your defined condition (see snippet above). The above snippet will now only create the resource in production.

Outputs & Imports​

Outputs allows you to export information of resource(s) that other stacks can them Import. This prevents us hard-coding things like ARN.

  PlatformEventBus:
    Type: AWS::Events::EventBus
    Properties:
      Name: !Sub ${AWS::StackName}-event-bus
      Tags:
        - Key: Environment
          Value: !Ref pEnvironment
        - Key: Project
          Value: !Ref AWS::StackName
        - Key: Role
          Value: event-bus
        - Key: Name
          Value: !Ref PlatformEventBus

  oPlatformEventBusName:
    Description: Platform event bus name
    Value: !Ref PlatformEventBus
    Export:
      Name: !Sub ${AWS::StackName}-services-event-bus-name
  oPlatformEventBusArn:
    Description: Platform event bus ARN
    Value: !GetAtt PlatformEventBus.Arn
    Export:
      Name: !Sub ${AWS::StackName}-services-event-bus-arn

The above snippet exports the ARN and name of the event bus.

Note: You'll notice to retrieve the name, we use !Ref PlatformEventBus but use !GetAtt PlatformEventBus.Arn to retrieve the ARN. The value a resource returns by !Ref varies, the docs for the resource will tell you what is returned by !Ref and what additional attributes can be accessed via !GetAtt`

The below snippet then shows a usage example where we are setting up an events trigger for a lambda. We import the Event Bus name using !ImportValue.

  Events:
    Trigger:
      Type: EventBridgeRule
      Properties:
        EventBusName: !ImportValue platform-infrastructure-services-event-bus-name
        Pattern:
          source:
            - customers
          detail-type:
            - OrganisationUpdated

Note: When a resource is being exported - you can't delete it or do any modification that will change the exported value as long as it is being imported (see here). Currently, this is fine as we are exporting stuff from our infrastructure stack that we don't want to change, and gives safety as these resources are being used. We are also exporting static stuff like ARNs which should not change.
If you are exporting resources that are more fluid and expect the actual output value to change, then you should use SSM to store these values and then export the ARN of the SSM parameters.

Useful Resources​