AWS Automation Using PowerShell - Part Un (Starting and Stopping EC2 Instances)

This is the first post in what will hopefully be a series of posts (good luck with that) on automating Amazon Web Services (AWS) deployments using PowerShell.


Pre-requisites


Before you get started, there are a few things that you need to do:

1. Gain basic understanding of AWS especially on EC2, S3, EBS, EIP and SNS.
2. Sign up for an AWS account.
3. Install PowerShell 2.0.
4. Install the latest version of the AWS SDK for .NET.
5. Create a directory in which you're going to create the PowerShell scripts.
6. Copy the AWSSDK.dll file from the bin directory of the SDK installation (e.g. C:\Program Files (x86)\AWS SDK for .NET\bin) to the new directory. This is to avoid having to deal with machine-specific installation paths and lets you deploy a self-contained set of automata. 
7. Buckle up for a joy ride ahead!

AWS SDK for .NET

The SDK comes with a Visual Studio 2008/2010 plugin that lets you create .NET projects that you can then easily deploy to AWS using a wizard-based GUI. On top of that, it also includes a .NET library (duh) that wraps calls to the AWS REST APIs.

Having said that, the IDE plugin is pretty much useless for our purposes since a good engineer always attempts to automate repetitive tasks, and using a GUI-based tool does not a good automaton make. Instead, we want to be able to write a program, or better yet, a script that can be executed from the command line interface. You can choose to write the automata in C#, but bear in mind that they will ultimately be owned and maintained by a DevOps team. Most DevOps engineers I know can't be bothered to learn C#, so PowerShell is probably be a more preferred option.


Spinning EC2 instances up and down

While the best practice in managing an AWS infrastructure is to leverage CloudFormation and deployment platforms like Puppet or Chef, doing so requires significant investment in understanding the technology and underlying concepts. What I'm going to do here is present a simple scenario where you need to start and stop specific EC2 instances so that we can gain basic understanding of AWS and its APIs before delving into more advanced scenarios. Think of it as getting your feet wet before diving in head first.

The following is a sample PowerShell script that starts and stops EC2 instances. Bear in mind that I'm a relative virgin when it comes to Powershell, having only been to the second base. So keep the laughter and scorn to yourself for now.

The code is actually pretty self-explanatory, but I'm going to walk you through it anyway.

1. The first line exposes four parameters, namely the operation (start or stop), the AWS account secret key, access key and a list of EC2 instance identifiers that you want to start or stop.
2. Line 3 is where you include a reference to the standalone copy of the SDK.
3. Lines 5-23 represent the meat of the script that starts and stops the instances. As you can probably gather, most AWS APIs follow the request-response pattern. You create a request to do something, send the request to AWS and get a response in return. It doesn't get any simpler than that.
4. Lines 25-29 refer to a function that creates and returns a 'proxy' to the EC2 API based on the specified secret and access keys.
5. Lines 31-41 contain a function that returns a generic list of EC2 instance identifiers based on the instanceIds parameter.
6. Lines 43-50 refer to a simple switch that executes either the start or stop function based on the operation parameter passed into the script.

You can then use the script as follows:

Go to your AWS console and lo and behold, the instances are spinning up!

There are many ways of automating this process. One option is to create builds in your favorite continuous integration platform (TeamCity, Go, TFS, etc.) that make use of this script to spin the instance up or down based on a specific schedule. Alternatively, you can configure the builds to not execute automatically so that you can manually trigger them with a single click of a button.

What's next?

The next step is to bootstrap the EC2 instances during startup so that each instance is responsible for configuring and installing its own resources dynamically. We can do fancy things like leveraging the Simple Notification Service (SNS) to notify interested parties (e.g. the build workflow) when the bootstrapping process is complete instead of polling each EC2 instance to see if it's up and running. This and more will be covered in the next post.

Till next time, adios.