Get a Customized Plan

The Fairway Technologies Blog

Immutable Services with Packer and AWS

In cloud, AWS, Packer, Docker No Comments
April 3, 2019

Just a few years ago, server-side software was dominated by heavyweight application servers. These servers ran on dedicated hardware, dedicated operating systems, and were managed by IT professionals. The process of adding new applications, or updating existing ones, depended on human interventions that could be difficult and error prone. Over time, the configuration would drift from a well-known state to an unknown state. This has been described as a snowflake server by Martin Fowler.

First, with virtualization technologies, and later with cloud- and container-based computing, a new paradigm has emerged. Tools now exist that allow developers to describe a service and its corresponding dependencies as a combination of configuration and code. With these tools, it is possible to package up the entirety of an application in a single immutable artifact. The advantage of this approach is often described in terms of Pets vs Cattle. The act of adding or removing instances of an immutable server is an expected and relatively painless operation. This, in sharp contrast to the earlier approach.

CloudNativeExplained

What follows here is a demonstration of using Packer to create an EC2 image for a simple Java Spring Boot-based web service. This is not a complete solution but is, rather, a technology demonstration to help illuminate some of the fundamental building blocks of this approach. Immutable servers are the first necessary step into the broader concern of immutable infrastructure. Not covered here are topics like infrastructure as code, continuous integration, continuous delivery, service discovery, distributed configuration, and many other related topics. Hopefully, we will be able to dig deeper into those topics at a later time.

To run this demo, you will require credentials as described here. For the purposes of this example, I created a local AWS profile and set the credentials for that profile.

aws configure --profile ImageBuilder

You will also need to install Packer and have a Java virtual machine installed to run the demo application.

The Spring Boot application was generated with Spring Initializr and it is located in the /src directory. The application has very little functionality. It just has an endpoint (http://localhost:8080/greeting?name=Dave) that will return a greeting. That should be enough functionality to prove that things are working!

Packer is a tool for creating machine images. It supports all kinds of image types, but for this demonstration, we will be focusing on AWS EC2 images. It should be noted that there are some other competing tools like BoxFuse in the market. Additionally, it is possible to script to the AWS API directly. In the world of containers, Dockerfiles serve a similar purpose. Packer seems to be the most widely used image creation tool and is supported by CD tools like Spinnaker.

Packer Templates are JSON files that describe how an image is produced. They can be arbitrarily named. For this demo, it is the file packer_build.json.

The first part of our build defines a variable that sets the image name.

  {
    ...
    "variables":{
      "ami_name":"Packer-Spring-Boot-Demo-{{isotime \"06-01-02-030405\"}}"
    }
    ...
  }

The next section is builders that define what kind of builder we are using—EC2 in this case. Part of this section includes some logic to define what base image to use for the build.

  {
    ...
    "source_ami_filter":{
      "filters":{
        "virtualization-type":"hvm",
        "name":"ubuntu/images/*ubuntu-bionic-18.04-amd64-server-*",
        "root-device-type":"ebs"
      },
      "owners":[
        "099720109477"
      ],
      "most_recent":true
    }
    ...
  }

This src_ami_filter will choose the latest 18.04 image published by Canonical.

The final section in our Packer template is provisioners. Provisioners are executed in order, and in this demo, four are defined.

The first one updates the system and applies any security patches.

Inside build.json 

    {
      "type":"shell",
      "scripts":[
        "packer-scripts/update-upgrade.sh"
      ],
      "pause_before":"4s"
    }

That is a shell provisioner that calls a script update-upgrade.sh.

#!/bin/bash -eux

sudo apt-get update
sudo unattended-upgrade -d

The next provisioner uploads the executable jar. For this demo, we are simply uploading the local build, you will need to run  ./gradlew assemble<  before running the Packer build. In a more realistic production environment, you would probably download the .jar file from an artifact repository or some other remote file store.

    {
      "type":"file",
      "source":"build/libs/packer-demo.jar",
      "destination":"/tmp/packer-demo.jar"
    }

The final two provisioners install and run an Ansible local provisioner.

This is the Ansible install. It uses the shell provisioner.

    {
      "type":"shell",
      "inline":[
        "sudo apt-get clean",
        "sudo apt-get update",
        "sudo apt-get install software-properties-common",
        "sudo apt-add-repository --yes --update ppa:ansible/ansible",
        "sudo apt-get -y install ansible"
      ]
    }

And this is the Ansible local provisioner.

    {
      "type":"ansible-local",
      "playbook_file":"ansible/playbook.yml",
      "playbook_dir":"ansible",
      "galaxy_file":"ansible/requirements.yaml"
    }

As you can see, the Ansible assets are located in the /ansible directory. It uses remyma.springboot to run the Spring Boot application.

To bring it all together, to run this demo.

Build the jar file.

$ ./gradlew assemble

Validate the template.

$ packer validate packer_build.json
Template validated successfully.

Build the template.

$ AWS_PROFILE='ImageBuilder' packer build packer_build.json

This will take some time and you will see shell output from the running scripts.

The final line will list the AMI id

==> Builds finished. The artifacts of successful builds are:
--> Packer Demo Spring Boot: AMIs were created:
us-east-1: ami-XXXXXXXXXXXXXXXX

You can then take a look at your shiny new AMI via the cli.

$ aws ec2 describe-images --image-ids ami-XXXXXXXXXXXXXXXX --profile ImageBuilder
   {
    "Images":[
        {
           "VirtualizationType":"hvm",
           "Name":"Packer-Spring-Boot-Demo-19-03-22-023850",
           "Hypervisor":"xen",
            "EnaSupport":true,
           "SriovNetSupport":"simple",
           "ImageId":"ami-XXXXXXXXXXXXXXXX",
           ...
        }
    ]
   }

You can clean up this demo by doing a deregister of the image imageand then deleting the snapshots.

Creating immutable machine images is an important first step to achieving immutable infrastructure. Packer can be an useful tool for creating those images. Integrating Packer into a more full-featured CI/CD pipeline is next logical step in moving towards that goal.

The source code for this project is available on bitbucket.

New Call-to-action
New Call-to-action