Amazon's Python AWS SDK, called boto3, includes an Amazon S3 client that enables access to AKAMAI CLOUD's Amazon S3-compatible Object Storage within a Python application or script.

📘

If you haven't already, review Data integrity protections.

Set up the AWS SDK for Python

Review these sections to get this software development kit (SDK) installed and configured for use with Object Storage.

Prerequisites

Consider and apply these points as necessary:

• You need Python 3.6 or later. Ensure at least this version is installed on your local system.

• You need a basic understanding of Python. You should be comfortable using it to integrate code samples into your application.

• You need a basic understanding of Object Storage. This includes how files are stored within a flat, non-hierarchical system alongside rich metadata. See the Overview, Endpoint types, and Quotas and limits topics for more information.

You need an Object Storage access key

This key grants AWS SDKs access to your Object Storage buckets and objects. See Manage access keys for instructions, if you don't already have one. Specifically, you'll need these values:

• The access key. After you create a key, a unique identifier is assigned to the key.

• The secret key. This is a unique secret that ​Akamai​ generates for the access key.

📘

The secret key is only displayed a single time, after you create the access key. So, if you don't know this value, you'll need to generate a new access key and be sure to store its secret key.

Install boto3

The first step is to install the AWS SDK for Python (boto3) through pip, Python's package manager, that's installed by default on recent versions of Python. Run this command in your project's directory:

pip3 install boto3

See the AWS SDK for Python's Quickstart guide for more information.

Initialize the client

To access Object Storage buckets and objects, you need to configure your credentials and initialize the Amazon S3 client session. To do this, you need to add this code to your python script, replacing these variable values:

• [access-key]. This is your Object Storage access key.
• [secret-key]. This is the secret key that ​Akamai Technologies, Inc.​ generated along with your Object Storage access key. If you don't know this value, you'll need to generate a new access key.
• [endpoint-url]. This is the endpoint URL that can be used to access all of your Object Storage resources within a certain region. See Supported endpoint types by region and locate your specific endpoint type's Endpoint (S3 hostname), based on its region. Prepend https:// to it to create the full [endpoint-url].

import boto3

linode_obj_config = {
    "aws_access_key_id": "[access-key]",
    "aws_secret_access_key": "[secret-key]",
    "endpoint_url": "[endpoint-url]",
}

client = boto3.client("s3", **linode_obj_config)

See the Boto3 Configuration guide for more details on how to configure the client.

🚧

If you need to share this code with others, hide both the [access-key] and [secret key]. There are a few ways to do this, including using AWS configuration files, environment variables through the shell, or a custom environment-specific configuration file.

Example operations

Here are some examples of common operations you can set up to interact with your buckets and objects.

List buckets

Lists all buckets in a region.

client.list_buckets()

This example lists all buckets on your account, in the region defined in your [endpoint-url]:

response = client.list_buckets()
for bucket in response['Buckets']:
    print(bucket['Name'])

👍

See list_buckets() for additional details, syntax, and examples.

Create a bucket

Create a new bucket you can use to store objects, by replacing bucket-label with a desired name.

client.create_bucket(
    Bucket='bucket-label')

📘

Bucket names have requirements

See Manage buckets for rules on naming the bucket.

This example creates a new bucket named "example-bucket," in the region defined in your [endpoint-url]:

client.create_bucket(Bucket='example-bucket')

👍

See create_bucket() for additional details, syntax, and examples.

Delete a bucket

Delete a specific, empty bucket by replacing bucket-label with the name of the target bucket:

client.delete_bucket(
    Bucket='bucket-label')

📘

You can only use this operation to delete a bucket if it's empty. If the bucket still contains objects, you need to delete them before you can delete the bucket.

This example deletes a bucket named "example-bucket":

client.delete_bucket(Bucket='example-bucket')

👍

See delete_bucket() for more details.

List objects

Lists all the objects in a specific bucket and optionally within a certain prefix. Replace these variable values:

• bucket-label. The name of the target bucket.
• object-prefix. The optional prefix (or pseudo path) for objects in a bucket, that you want to list. Leave this out to list all objects.

client.list_objects(
    Bucket='bucket-label',
    Prefix='object-prefix')

This example lists all objects in a bucket named "example-bucket":

response = client.list_objects(Bucket='example-bucket')
for object in response['Contents']:
    print(object['Key'])

This example lists all objects stored in the "assets/" folder within the bucket named "example-bucket". Keep in mind that objects aren't actually stored in folders, but the prefix value allows them to appear within a structure:

response = client.list_objects(Bucket='example-bucket', Prefix='assets/')
for object in response['Contents']:
    print(object['Key'])

👍

See list_objects() for additional details, syntax, and examples.

Upload a file as an object

Uploads a file as an object to be stored in a specific bucket. Replace these variable values:

• bucket-label. The name of the target bucket.
• object-name. The name of the object you wish to create, including any prefix/path.
• path/to/file.ext. The full path and filename of the file you want to upload.

client.upload_file(
    Filename='/path/to/file.ext',
    Bucket='bucket-label',
    Key='object-name')

This example uploads the file "file.txt", which is located in the same directory as your Python script, to "example-bucket". It names this new object "file.txt", using the same as the filename:

client.upload_file(Filename='file.txt', Bucket='example-bucket', Key='file.txt')

This example uploads the file "logo.jpg" that's located in your home folder, to "example-bucket". It names this new object "images/logo.jpg", which allows it to be structured within a pseudo folder:

client.upload_file(Filename='/Users/user/logo.jpg', Bucket='example-bucket', Key='images/logo.jpg')

👍

See upload_file() for additional details, syntax, and examples.

Download an object to a file

Downloads the specified object to a new file on your system. Replace these variable values:

• bucket-label. The name of the bucket where the target object is stored.
• object-name. The name of the object you want to download, including any prefix/path.
• /path/to/new/file.ext. The path and new filename for the new object that will be created in your bucket.

client.download_file(
    Bucket='bucket-label',
    Key='object-name',
    Filename='/path/to/file.ext')

This example downloads the object "file.txt", that's stored in "example-bucket" to live as "file.txt" in the same folder as your Python script:

client.download_file(Bucket='example-bucket', Key='file.txt', Filename='file.txt')

👍

See download_file() for additional details, syntax, and examples.

Delete an object or directory

Deletes an object from a bucket. Replace these variable values:

• bucket-label. The name of the bucket where the target object is stored.
• object-name. The name of the object you want to delete, including any prefix/path.

client.delete_object(
    Bucket='bucket-label',
    Key='object-name')

The example deletes the object "file.txt", that's stored in "example-bucket":

client.delete_object(Bucket='example-bucket', Key='file.txt')

👍

See delete_object() for additional details, syntax, and examples.