Chef enables you to automate your infrastructure. It provides a command line tool called knife to help you manage your configurations. Using the knife EC2 plugin you can manage your Amazon EC2 instances with Chef. knife EC2 makes it possible to create and bootstrap Amazon EC2 instances in just one line – if you go through a few setup steps. In this article, I want to show you how to setup your Chef installation and AWS configuration so that you can easily bootstrap new Amazon EC2 instances with Chef’s knife.
Prepare SSH Access To Your Amazon EC2 Instances
Configure Your Amazon Security Group
As Amazon blocks all incoming traffic to your EC2 instances by default. you’ll need to open the SSH port for
knife to access a newly created instance. This is pretty easy. Just login to the AWS management console and navigate to EC2 under Services > Compute. Go to Security Groups and select the default group.
Open the Actions drop-down and choose Edit inbound rules:
Then add a rule for Type
SSH with Source
Anywhere and save the new inbound rule:
This enables SSH connections from anywhere (0.0.0.0/0) in the world. If you want to limit access to just your home or work network, choose
Custom IP instead of
Anywhere and enter the corresponding net mask.
When you’re done, your default security group should look like this:
Generate Key Pair in AWS Console
To enable SSH access to your Amazon EC2 instances you need to create a key pair. Amazon will install the public key of that key pair on every EC2 instance. knife will use the private key of that key pair to connect to your Amazon EC2 instances.
Just select Key Pairs under NETWORK & SECURITY in your AWS management console and press Create Key Pair. Give it a name (e.g. knife so you know that this key pair will be used by knife) and press Create. This will create the key pair and download the private key to your local workstation.
Store the downloaded private key knife.pem in
Prepare your SSH configuration to avoid host key mismatch errors
~/.ssh/config and add:
Host ec2*compute-1.amazonaws.com StrictHostKeyChecking no User ubuntu IdentityFile /Users/mm/.ssh/knife.pem
(make sure you fix the path to your home dir)
Now, SSH access to your Amazon EC2 instances will work. Time to move on to the next step.
Configure knife Enabling it to Manage EC2 Instances
Install the knife EC2 plugin
To enable knife to manage Amazon EC2 instances you need to install the knife EC2 plugin.
Either add it to your
Gemfile or install it using
$ gem install knife-ec2
The knife EC2 plugin adds the
ec2 sub-command to knife, which we’ll use to manage our Amazon EC2 instances.
Tell knife about your AWS credentials
Create a new user for knife in your AWS management console under Services > Administration & Security. You’ll use this user’s AWS credentials (access key ID and secret access key) to manage your Amazon EC2 instances with knife EC2.
Add the AWS credentials of your knife user to your knife configuration file
knife[:aws_access_key_id] = "..." knife[:aws_secret_access_key] = "........."
Now, knife should be able to use the Amazon AWS API to manage Amazon EC2 instances.
Choose an AMI for your Amazon EC2 instances
We’re going to instantiate an Amazon EC2 instance running Ubuntu 14.04 LTS.
Look here for Ubuntu 14.04 LTS AMI IDs which you can use to instantiate Amazon EC2 instances with knife.
You need to choose the right AMI for your region, architecture and root storage. Note down the AMI ID (ami-XXXXXXXX) to use it with knife.
Create an EC2 instance using Chef knife
Now, it’s time to use knife to fire up and configure a new Amazon EC2 instance.
$ knife ec2 server create -r "role[ubuntu]" -I ami-1ed88f69 -f t2.small -S knife -i ~/.ssh/knife.pem --ssh-user ubuntu --region eu-west-1 -Z eu-west-1a
"role[ubuntu]"is the run_list I want to associate with the newly created node. You can put any roles and recipes you like here
-Iis the AMI ID you selected earlier
-fis the Amazon EC2 instance type (see Model)
-Sis the name you gave to the SSH key pair generated in the AWS management console
-ipoints to the private key file of that SSH key pair as downloaded when the key pair was created in the AWS management console
--ssh-userthe official Ubuntu EC2 AMIs use ubuntu as the default user
--region eu-west-1If you want your instances to be deployed in any specific Amazon AWS region, add this parameter and the desired region
-Z eu-west-1ais the availability zone within your region
ATTENTION: make sure to kill the instance again if not needed anymore 😉
Managing Amazon EC2 Instances With knife
Once you’ve started up at least one Amazon EC2 instance with knife, you can use knife to find running EC2 instances like this:
$ knife ec2 server list --region eu-west-1
(make sure you use the correct
And, if you want to get rid of an instance (terminate instance and delete the corresponding Chef node), it’s as easy as:
$ knife ec2 server delete i-XXXXXXXX --region eu-west-1 $ knife node delete i-XXXXXXXX
i-XXXXXXXX is the ID of the instance as found in the AWS management console or a
knife ec2 server list call)
After getting the initial setup right it’s a breeze to start, list, and stop your EC2 instances at Amazon with Chef’s knife ec2. With just a single command you can instantiate a new server, bootstrap it as a Chef client and run all Chef recipes defined in the run_list. Pretty sweet. What are your experiences with knife and Amazon EC2 (or other cloud providers)? Let us know in the comments…