Using AWS CLI (and AWS SDK)
Photo by Matt Artz on Unsplash
  • The AWS CLI is an open source tool built using the AWS SDK for Python (Boto3) that provides commands to interact with AWS services. With minimal configuration, you can use all the features provided by the AWS management console from your favorite terminal.

  • AWS SDKs provide an API for different programming languages (Python, Java, JavaScript, C++, .NET, GO, PHP, Ruby,…) to programmatically build and use AWS services.

In this article, we will see some tips to make the best use of these 2 tools.

CLI AWS Installation

AWS CLI is available in 2 versions:

  • Version 2: the latest and supports the latest features
  • Version 1: original version, should no longer be used

To install AWS CLI Version 2 on Docker, Linux, macOS or Windows, refer to the AWS documentation

After a successful installation, you should be able to run the following commands:

% which aws

% aws --version
aws-cli/2.2.16 Python/3.8.8 Darwin/20.6.0 exe/x86_64 prompt/off

AWS CLI Profiles

It is possible to register multiple user accounts in AWS CLI. This is called Profiles.

To do this, once your default account is configured, you can add a new one by executing the command:

% aws configure --profile <name-of-other-profile>

Now you can run an AWS CLI command under this new Profile by adding the parameter:

% aws iam get-user --profile <name-of-other-profile>

Temporary session

When an AWS account is protected by an MFA code, it is possible to create a temporary session using AWS Security Token Service (AWS STS) to request temporary identifiers with limited privileges.

To do this, run the following command:

% aws sts get-session-token --serial-number <arn-of-your-mfa-device> --token-code <token-code-from-your-mfa-device> --duration-seconds <session-duration-in-seconds>

    "Credentials": {
        "AccessKeyId": "ASIH9FS4N0N76VVTDFDG5C",
        "SecretAccessKey": "4h0f4j88H9L/HT4I0xjvE9-jrf72jkISjklbjEvRHfn",
        "SessionToken": "EJu6g3JpZ2luX2VjEDoaCWV.../...wvA5a5mfFWUOrH62fDFYbmUW5j31k1r/igIgP14W374njGO+mbO5+MeYKGoaLvHAC0SFdXh9sjopz2wq7gEIQxABGgw3OTc1MTc4.../...k+4b6sGGym82jlaPI=",
        "Expiration": "2021-07-26T10:10:59+00:00"

You get new Credentials, valid, the time of the session.

Priorities of Credentials

It is possible to set Credentials in several places. There is therefore a priority order that must be known to fully understand the adverse effects that this can cause.


  1. Passed in command line (–region, –output, –profile)
  3. Saved in the /. aws/credentials file generated by the aws configure command
  4. Saved in the /. aws/config­ file generated by the aws configure command
  5. Recorded in Container Credentials (for ECS Tasks)
  6. Saved in the EC2 Instances Profiles


  1. Passed in Language System Properties
  3. Saved in the default /. aws/credentials file in many SDKs
  4. Recorded in Container Credentials (for ECS Tasks)
  5. Saved in the EC2 Instances Profiles

Good Practices

To avoid any pitfalls with Credentials, here are some rules to follow:

  1. NEVER save Credentials in the code!!
  2. But rather define the Credentials in the best place in the priority chain:
    • If you are inside AWS, use Roles IAM (EC2 Instance Roles, ECS Roles, Lambda Roles)
    • If you are outside the AWS network, use environment variables or Profiles

Signing HTTP Request

When using AWS CLI or AWS SDK, HTTP calls to AWS are automatically signed. The protocol used is called Signature Version 4 (SigV4) and comes from AWS.

It comes in two possible forms:

  • HTTP headers (Authorization header)
  • URL parameters (Query string)


Here are some tools that will help you debug and understand what’s going on if there are problems:

Policy Simulator

It may be useful to check the access rights to an AWS resource based on a User, Group or Role. There is a tool in AWS that can run these tests, the Policy Simulator:

Dry Run

It may also be useful to test an AWS CLI command by simulating its execution. AWS CLI commands have an option for this:

  • Example of simulated creation of an EC2 instance:
% aws ec2 run-instances --dry-run --region eu-west-3 --image-id ami-062fdd189639d3e93 --instance-type t2.micro

An error occurred (DryRunOperation) when calling the RunInstances operation: Request would have succeeded, but DryRun flag is set.

Because the command is run in dry-run mode, if successful, it returns DryRunOperation. If failed, it would return UnauthorizedOperation.


Some AWS CLI commands return a encoded authorization message describing the problem. This message must be decoded to be understandable.

To do this, you can use the command:

% aws decode-authorization-message --encoded-message <encoded-message>

AWS EC2 Instance Metadata

Instances Metadata are data relating to an EC2 instance: they are accessible from the instance and allow you not to have to use IAM Role since this data has already been loaded into the instance for its configuration or creation.

They are available at:

Note that this is a local address and therefore only accessible from the EC2 instance.

Examples of use

Data Types

ec2-user ~ % curl

There are 3 types of data accessible from an EC2 instance as we can see in the output of this command.


You can access the instance metadata from a running instance using one of the following methods:

  • Instance Metadata Service Version 1 (IMDSv1) – request/response method
  • Instance Metadata Service Version 2 (IMDSv2) – session-oriented method

When using session-oriented requests (IMDSv2), you create a session token that defines the duration of the session, which must be a minimum of one second and a maximum of six hours. During the specified period, you can use the same session token for subsequent requests.

Here’s how to retrieve a session token:

ec2-user ~ % TOKEN=`curl -X PUT "" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"`

You can then use it in the following commands, the time of the session:

ec2-user ~ % curl -H "X-aws-ec2-metadata-token: $TOKEN" -v

Some instance metadata

Here is a description of some metadata of instances that may be useful:

Path de la MetadataDescription
ami-idThe AMI ID used to launch the instance.
ami-launch-indexIf you started more than one instance at the same time, this value indicates the order in which the instance was launched. The value 0 indicates the first instance launched.
block-device-mapping/amiThe virtual device that contains the root/boot filesystem.
block-device-mapping/ebsNVirtual devices associated with any Amazon EBS volume. Amazon EBS volumes are only available in the metadata if they were present at the time of launch or when the instance was last started. The N indicates the Amazon EBS volume index (such as ebs1 or ebs2).
events/recommendations/rebalanceApproximate time, UTC, at which the rebalancing recommendation notification EC2 is issued for the instance. Here is an example of metadata for this category : {"noticeTime": "2020-11-05T08:22:00Z"}. This category is only available after the notification has been issued.
hostnameThe private IPv4 DNS host name of the instance. In case of multiple network interfaces are present, this refers to the eth0 device (the device whose device number is 0).
iam/infoIf an IAM role is associated with the instance, it contains information about the last update of the instance profile, among which the date of last update up-to-date (LastUpdated), InstanceProfileArn and InstanceProfileId of the instance. Otherwise, absent.
iam/security-credentials/role-nameIf an IAM role is associated with the instance, role-name is the name of the role and role-name contains the temporary security identification information associated with the role. Otherwise, absent.
instance-idThe ID of this instance.
instance-typeThe type of instance.
kernel-idThe kernel ID launched with the instance, if applicable.
local-hostnameThe private IPv4 DNS host name of the instance. In case of multiple network interfaces are present, this refers to the eth0 device (the device whose device number is 0).
local-ipv4The private IPv4 address of the instance. In case several network interfaces are present, this refers to the eth0 device (the device whose number of device is 0).
network/interfaces/macs/mac/ipv4-associations/public-ipPrivate IPv4 addresses that are associated with each public IP address and assigned at this interface.
network/interfaces/macs/mac/local-hostnameThe local host name of the interface.
network/interfaces/macs/mac/local-ipv4sPrivate IPv4 addresses associated with the interface.
network/interfaces/macs/mac/public-hostnameThe public DNS of the interface (IPv4). This category is only returned if the enableDnsHostnames is defined as true.
placement/availability-zoneThe availability zone in which the instance was launched.
placement/regionAWS region where the instance is launched.
public-hostnameThe public DNS of the proceeding. This category is only returned if the enableDnsHostnames attribute is set as true.
public-ipv4The public IPv4 address. If an Elastic IP address is associated with the instance, the returned is the IP Elastic address.

The names of the security groups applied to the proceeding.

After launch, you can change the security groups of the instances. From such changes appear here and in network/interfaces/macs/mac/security groups.


An AWS SDK (Software Development Kit) is used to interact with AWS within an application. There are therefore many AWS SDKs depending on the different programming languages (Python, Java, C++, JavaScript, Ruby, PHP,…):

  • The AWS CLI is itself encoded from the Python AWS SDK (also called Boto3).
  • Some AWS services are only accessible by an AWS SDK: DynamoDB, Lambda Function,…

Note that if you do not have a default Region, the AWS SDKs will interact with the default Region us-east-1.

AWS Limits and Backoff

Limits / Quotas

There are Limits or Quotas in AWS that you must be aware of:

  • API Rate Limits: depending on the AWS APIs, one cannot make more than a certain number of calls per second to an API
  • Service Quotas (or Service Limits): According to the AWS Services, you cannot consume more than a certain number of AWS Services (e.g. 1152 vCPU per AWS account)

Exponential Backoff

When you get ThrottlingException errors when calling AWS Services, you must use the Exponential Backoff.

It is a retry mechanism with a duration between each attempt that increases exponentially. It is described more precisely in this article

  • AWS SDKs already implement it, so there is nothing to do
  • But if you make calls to the AWS APIs by another means, you MUST set up such a mechanism:
    • In case of ThrottlingException or 5xx errors
    • Not in case of 4xx errors
Jean-Jerome Levy

Written by

Jean-Jerome Levy

DevOps Consultant

Seasoned professional in the field of information technology, I bring over 20 years of experience from working within major corporate IT departments. My diverse expertise has played a pivotal role in a myriad of projects, marked by the implementation of innovative DevOps practices.