This is a demo of the features of AWS Boto3 SDK for EC2

In this demo we show

  1. how to look at the status of our current instances
  2. how to create new instances
  3. how to stop and delete them
  4. how to attach an EBS volume to an instance
  5. how to attach an NFS volume to an instance

Items 4 and 5 require access to ssh, so this won't run on Windows, but it works fine on the Mac and Linux.

there are a bunch of instance ids and volume ids here, but they have been deleted. you will need to create your own.

In [1]:
import boto3

creating an instance

first you need you aws_access_key_id and secret_access_key that were created when you created your account

In [2]:
ec2 = boto3.resource('ec2', 'us-west-2',
        aws_access_key_id = 'your access key',
        aws_secret_access_key = 'your big long secret key')

The following function is a handy way to see the status of your instances

In [3]:
def show_instance(status):
    instances = ec2.instances.filter(
        Filters=[{'Name': 'instance-state-name', 'Values': [status]}])
    for instance in instances:
        print(, instance.instance_type, instance.image_id, instance.public_ip_address)
In [4]:
('i-0ccfc93aaf3e0305c', 't2.large', 'ami-d0f506b0', None)
('i-024a5f16ce5b1ed02', 't2.medium', 'ami-7172b611', None)

this will start a stopped instance

In [5]:
[{'ResponseMetadata': {'HTTPHeaders': {'content-type': 'text/xml;charset=UTF-8',
    'date': 'Fri, 23 Sep 2016 19:10:42 GMT',
    'server': 'AmazonEC2',
    'transfer-encoding': 'chunked',
    'vary': 'Accept-Encoding'},
   'HTTPStatusCode': 200,
   'RequestId': '1e3521ef-bc44-42c2-9261-b7528e071967',
   'RetryAttempts': 0},
  u'StartingInstances': [{u'CurrentState': {u'Code': 0, u'Name': 'pending'},
    u'InstanceId': 'i-0ccfc93aaf3e0305c',
    u'PreviousState': {u'Code': 80, u'Name': 'stopped'}}]}]
In [6]:
('i-0ccfc93aaf3e0305c', 't2.large', 'ami-d0f506b0', '')
('i-0a184b56b0ebdba98', 't2.micro', 'ami-7172b611', '')

This is how you can stop and terminate an instance.

In [9]:
stoplist = ['i-0ccfc93aaf3e0305c']
[{'ResponseMetadata': {'HTTPHeaders': {'content-type': 'text/xml;charset=UTF-8',
    'date': 'Fri, 23 Sep 2016 19:14:39 GMT',
    'server': 'AmazonEC2',
    'transfer-encoding': 'chunked',
    'vary': 'Accept-Encoding'},
   'HTTPStatusCode': 200,
   'RequestId': '7092e9ac-8cf3-4d0c-87b8-556fedcfc459',
   'RetryAttempts': 0},
  u'StoppingInstances': [{u'CurrentState': {u'Code': 64, u'Name': 'stopping'},
    u'InstanceId': 'i-0ccfc93aaf3e0305c',
    u'PreviousState': {u'Code': 16, u'Name': 'running'}}]}]

if i want to permanently delete an instance i can terminate it with


To create an instance you need a keypair.

This is easiest to do from the portal assuming you have a keypair called escience1 here is the way to create an instance.

In this case we are using MaxCount = 1. If we had MaxCount = 5 it will try to create 5 instances for us.

In [10]:
ec2.create_instances(ImageId='ami-7172b611', InstanceType='t2.micro', KeyName='escience1', MinCount=1, MaxCount=1)
In [11]:
('i-0775b0a28a1be4ed0', 't2.micro', 'ami-7172b611', '')
('i-0a184b56b0ebdba98', 't2.micro', 'ami-7172b611', '')
In [12]:
import subprocess
import sys
In [15]:
vols = ec2.volumes.filter(
for vol in vols:
    print(, vol.size, vol.state)
('vol-05731419e6abb49a9', 8, 'in-use')
('vol-032807a231219af70', 8, 'in-use')
('vol-0bdd0584d0833e691', 20, 'available')
('vol-07ce6f03c1a13d5a7', 100, 'in-use')
('vol-0ce3df91d4d2e07e0', 8, 'in-use')
('vol-0fc1ff8711cd0eac4', 8, 'in-use')

To attach the volume to an instance we first create a volume object and then attach it to an instance.

It is important to note that the volume and the instance must be in the same availability zone.

In [18]:
vol = ec2.Volume('vol-0bdd0584d0833e691')
In [20]:
{u'AttachTime': datetime.datetime(2016, 9, 23, 20, 49, 39, 637000, tzinfo=tzutc()),
 u'Device': '/dev/xvdh',
 u'InstanceId': 'i-0a184b56b0ebdba98',
 'ResponseMetadata': {'HTTPHeaders': {'content-type': 'text/xml;charset=UTF-8',
   'date': 'Fri, 23 Sep 2016 20:49:38 GMT',
   'server': 'AmazonEC2',
   'transfer-encoding': 'chunked',
   'vary': 'Accept-Encoding'},
  'HTTPStatusCode': 200,
  'RequestId': '085a8a8e-5296-4912-969a-2fe1c05809bc',
  'RetryAttempts': 0},
 u'State': 'attaching',
 u'VolumeId': 'vol-0bdd0584d0833e691'}

Mounting the new volume on an instance

Mounting a volume on a file system cannot be done with boto3 directly because the mount commands must be executed by the operating system. However we can use SSH to connect to the instance and execute the commands remotely.

The following function uses Python to create a subprocess to invoke ssh. Unfortunately this will only work on Linux or the Mac OS, because SSH is not a regular shell command for Windows. What follows was executed on a Mac.

In [21]:
def myexec( pathtopem, hostip, commands):
        ssh = subprocess.Popen(['ssh', '-i', pathtopem, 'ec2-user@%s'%hostip, commands ],
        result = ssh.stdout.readlines()
        if result == []:
                error = ssh.stderr.readlines()
                print >>sys.stderr, "ERROR: %s" % error
                return "error"
                return result

Before we can mount the volume we must make sure it has a file system. Then we can create a directory at the root called "/data" as the mount point. then we do the mount and run a "df" to see if it is there. we will need our private key to accomplish this.

In [60]:
priv_key = 'path-to-your-secret-key.pem'
In [22]:
command = 'sudo mkfs -t ext3 /dev/xvdh\n \
           sudo mkdir /data\n \
           sudo mount /dev/xvdh /data\n \
In [23]:
myexec(priv_key, '', command)
['Filesystem     1K-blocks    Used Available Use% Mounted on\n',
 '/dev/xvda1       8123812 1211172   6812392  16% /\n',
 'devtmpfs          501092      60    501032   1% /dev\n',
 'tmpfs             509668       0    509668   0% /dev/shm\n',
 '/dev/xvdh       20511356   45128  19417652   1% /data\n']

As you can see our 20G volume is now available as "/data". Let's double check

In [24]:
vols = ec2.volumes.filter(
for vol in vols:
    print(, vol.size, vol.state)
('vol-05731419e6abb49a9', 8, 'in-use')
('vol-032807a231219af70', 8, 'in-use')
('vol-0bdd0584d0833e691', 20, 'in-use')
('vol-07ce6f03c1a13d5a7', 100, 'in-use')
('vol-0ce3df91d4d2e07e0', 8, 'in-use')
('vol-0fc1ff8711cd0eac4', 8, 'in-use')

Now let's make a subdirectory for user ec2-user called mydata and create a file.

In [ ]:
command = 'cd /data\n \
           sudo mkdir mydata\n \
           sudo chown ec2-user mydata\n \
           cd mydata\n \
           touch filex\n \
           ls -l\n'
In [26]:
myexec(priv_key, '', command)
['total 0\n', '-rw-rw-r-- 1 ec2-user ec2-user 0 Sep 23 20:51 filex\n']

mounting an NFS file system.

EBS volumes can only be mounted on one instance at a time, though they can be detached and attached and mounted on other instances.

The AWS EC2 Elastic File System (EFS) provides a brilliant way to attach and mount an filesystem that satifies the Networked File System (NFS) standards.

We first need to make sure that the instance has the right type of security group. A security group defines the network protocols. We need to make sure that we have one in which port 2049 (NFS) is open.

We also need to know a few things about the instance i-0a184b56b0ebdba98 which was created from the portal as described in the book text example. When we created this instance we gave it a special security group "default" and we added the NFS 2049 port. we also need to know the subnet number.

In [55]:
instances = ec2.instances.filter(
    Filters=[{'Name': 'instance-state-name', 'Values': ['running']}])
for instance in instances:
        print(, instance.instance_type, instance.subnet_id, instance.security_groups)
('i-0a184b56b0ebdba98', 't2.micro', 'subnet-06e66170', [{u'GroupName': 'default', u'GroupId': 'sg-c67ce2a0'}])
In [56]:
ec2_client = boto3.client('ec2', 'us-west-2',
        aws_access_key_id = 'your access key',
        aws_secret_access_key = 'your big long secret key')
In [57]:
{'ResponseMetadata': {'HTTPHeaders': {'content-type': 'text/xml;charset=UTF-8',
   'date': 'Fri, 23 Sep 2016 22:32:47 GMT',
   'server': 'AmazonEC2',
   'transfer-encoding': 'chunked',
   'vary': 'Accept-Encoding'},
  'HTTPStatusCode': 200,
  'RequestId': '5acbe7db-579a-444b-8465-2500f55ff78a',
  'RetryAttempts': 0},
 u'SecurityGroups': [{u'Description': 'default VPC security group',
   u'GroupId': 'sg-c67ce2a0',
   u'GroupName': 'default',
   u'IpPermissions': [{u'FromPort': 22,
     u'IpProtocol': 'tcp',
     u'IpRanges': [{u'CidrIp': ''}],
     u'PrefixListIds': [],
     u'ToPort': 22,
     u'UserIdGroupPairs': []},
    {u'FromPort': 2049,
     u'IpProtocol': 'tcp',
     u'IpRanges': [],
     u'PrefixListIds': [],
     u'ToPort': 2049,
     u'UserIdGroupPairs': [{u'GroupId': 'sg-c67ce2a0',
       u'UserId': '066301190734'}]}],
   u'IpPermissionsEgress': [{u'IpProtocol': '-1',
     u'IpRanges': [{u'CidrIp': ''}],
     u'PrefixListIds': [],
     u'UserIdGroupPairs': []}],
   u'OwnerId': '066301190734',
   u'VpcId': 'vpc-d9a3dfbd'}]}

we can not start building the EFS file_system. this requires an efs client. then we can create the file system

In [30]:
client = boto3.client('efs','us-west-2',
        aws_access_key_id = 'your access key',
        aws_secret_access_key = 'your big long secret key')
In [32]:
response = client.create_file_system(
In [33]:
{u'CreationTime': datetime.datetime(2016, 9, 23, 14, 34, 41, tzinfo=tzlocal()),
 u'CreationToken': u'myefs',
 u'FileSystemId': u'fs-69fe00c0',
 u'LifeCycleState': u'creating',
 u'NumberOfMountTargets': 0,
 u'OwnerId': u'066301190734',
 u'PerformanceMode': u'generalPurpose',
 'ResponseMetadata': {'HTTPHeaders': {'content-length': '251',
   'content-type': 'application/json',
   'date': 'Fri, 23 Sep 2016 21:34:41 GMT',
   'x-amzn-requestid': '894d999c-81d5-11e6-8f18-d920b604e82f'},
  'HTTPStatusCode': 201,
  'RequestId': '894d999c-81d5-11e6-8f18-d920b604e82f',
  'RetryAttempts': 0},
 u'SizeInBytes': {u'Value': 0}}

Next step we must create a mount target

In [40]:
mtresp = client.create_mount_target(
In [41]:
{u'FileSystemId': u'fs-69fe00c0',
 u'IpAddress': u'',
 u'LifeCycleState': u'creating',
 u'MountTargetId': u'fsmt-ba6d9413',
 u'NetworkInterfaceId': u'eni-0cda9e42',
 u'OwnerId': u'066301190734',
 'ResponseMetadata': {'HTTPHeaders': {'content-length': '208',
   'content-type': 'application/json',
   'date': 'Fri, 23 Sep 2016 21:57:00 GMT',
   'x-amzn-requestid': 'a70fd33a-81d8-11e6-a3a9-91080f5279cd'},
  'HTTPStatusCode': 200,
  'RequestId': 'a70fd33a-81d8-11e6-a3a9-91080f5279cd',
  'RetryAttempts': 0},
 u'SubnetId': u'subnet-06e66170'}

The rest of the install requires shell commands to mount the file system

we first must install the nfs utilities.

next we create a mount point. we will call it /scidata

finally we need to do the mount. We can use the IP address of the mount target for that one.

In [63]:
command = 'sudo yum install -y nfs-utils'
In [64]:
myexec(priv_key, '', command)
['Loaded plugins: priorities, update-motd, upgrade-helper\n',
 'Package 1:nfs-utils-1.3.0-0.21.amzn1.x86_64 already installed and latest version\n',
 'Nothing to do\n']
In [69]:
command = 'sudo mkdir /scidata\n ls -l / | grep scidata \n'
myexec(priv_key, '', command)
['drwxr-xr-x  2 root root  4096 Sep 23 22:45 scidata\n']
In [73]:
command = 'sudo mount -t nfs4 -o vers=4.1  /scidata \n \
           df \n'
mntcmd = myexec(priv_key, '', command)
In [74]:
['Filesystem             1K-blocks    Used        Available Use% Mounted on\n',
 '/dev/xvda1               8123812 1211296          6812268  16% /\n',
 'devtmpfs                  501092      64           501028   1% /dev\n',
 'tmpfs                     509668       0           509668   0% /dev/shm\n',
 '/dev/xvdh               20511356   45128         19417652   1% /data\n',
 ' 9007199254740992       0 9007199254740992   0% /scidata\n']
In [ ]: