Object Storage is Hypertext Transfer Protocol (HTTP)-accessible and designed for building cloud-native applications that require scale and flexibility, as well as storing application artifacts such as analytics, backups, and archives.

You can combine Object Storage on Akamai Cloud with the Akamai content delivery network (CDN) to create a viable solution for applications that serve large files and datasets. This includes video-on-demand (VOD) streaming, e-commerce, firmware update distribution, media-based website content, and other applications.

Here, we focus on architectural best practices for designing and operating reliable, secure, efficient, and sustainable content storage and delivery systems using Object Storage on Akamai Cloud with the Akamai CDN.

Object Storage and data delivery

With the right bucket architecture, you can use Object Storage to house content for effective, unstructured data delivery. Object Storage supports critical features such as encryption, compression, deduplication, and versioning.

Its accessibility via HTTP protocols offers direct access to objects. Object Storage doesn't require an intermediary proxy to serve files—like a Block Storage volume attached to a Linode—and it can be accessed by a wide range of APIs.

These factors make Object Storage ideal for storing unstructured data like video and audio files that don't require frequent updating, and services that require efficient input and output.

Get to know the environment

To begin, review these sections to understand how combining Object Storage with the Akamai CDN can benefit you.

Here, we cover the specifications, considerations, and strategies for streaming data with Object Storage.

Technical specifications

When architecting applications that source data from Object Storage—like streaming video or audio—consider these technical factors:

Bandwidth and latency. Object Storage solutions are not always optimized for high-bandwidth streaming applications. This can lead to potential buffering issues and slower delivery speeds.
Scalability. To preserve infrastructure integrity, Object Storage systems may have limitations in place for concurrent connections and simultaneous user streams. This can impact the ability to efficiently handle spikes in demand.
Retrieval speed. Without CDN edge-based caching, content streamed directly from Object Storage origin servers may result in slower data retrieval speeds and higher latency.
Data transfer and egress costs. Streaming directly from Object Storage can incur high egress costs, because each user request may result in data transfer fees. When using a CDN, only requests to update the CDN cache may incur data transfer fees.
Resource limits. Object Storage systems may have limitations on the size of individual objects, potentially restricting the size of videos that can be efficiently streamed.
Rate limits. Object Storage solutions often have rate limits imposed on a requests per second (RPS) basis. If your users are accessing Object Storage endpoints directly without the use of a CDN, there's a greater chance these rate limits are reached as more users access content.
Updating manifests. You can determine bucket contents by syncing with a manifest. This means that when source manifests are updated often, buckets may constantly have their contents changed.

Network considerations

There are multiple internet-related factors to consider when building out a content delivery solution:

First mile. The content provider connection point. Should content providers set up content to be accessible from a single physical location, like an origin, user access is constrained by first mile connectivity limitations.
Last mile. The end-user connection point. Slow user internet connection speeds are uncontrollable and unpredictable. They can also mask or obfuscate problems related to the first mile, network peering, and bottleneck issues.
Peering points. The connection points between two networks. These connections are not guaranteed and are difficult to troubleshoot. Networks benefit financially from connectivity with content providers and end users but not each other.
Backbone. The infrastructure and physical connections that make up the internet. Hardware and network capacity limitations can cause performance issues.

Region-based architecture (only using Object Storage)

Without the use of a CDN, Object Storage content distribution is limited to individual, origin regions. Region-based delivery relies heavily on public networks to pull and deliver content from regional origin points. At scale, this may result in bandwidth issues, high latency, and inefficient experiences for large numbers of users.

Optimize your bucket architecture

Your Object Storage bucket architecture is critical to performance success. In particular, distributing content across multiple buckets helps with load distribution, CDN optimization, and adds security benefits like segmentation and origin obfuscation.

Here, we'll walk through bucket design strategies using a commerce site example, including an optimal bucket architecture for Akamai CDN integration.

Bucket architecture strategy

Object Storage stores files in a "flat" or unstructured file structure. Bucket contents don't exist in a hierarchy like traditional file structures. But, you can emulate hierarchy by creating folders within a bucket. Files (or objects) are stored alongside their rich metadata, and access can be granted on a per-object level, with each object assigned a unique URL.

For the example bucket architecture, consider this scenario:

An online commerce site serving audio, video, and image-based content.
All audio, video, and image files are stored on object storage as an origin point.

Don't use a single-bucket architecture

Let's assume you have a single-bucket architecture with a top-level directory named store, that has paths to three lower-level directories containing different types of content: music, video, and images. This directory and its sub-directories are in a single bucket. This bucket has a defined requests per second (RPS) limit, which is shared by all objects in the bucket.

Paths in this bucket are: store/music, store/video, and store/images.

Additional content such as copyrighted music or free music is placed as sub-directories under the store/music directory. Paths in this bucket now include: store/music, store/music/copyright, store/music/free, store/video, and store/images.

This architecture places content paths and object endpoints all within a single point of access—the store bucket, creating a bottleneck. You'll more than likely see poor performance here. Rate limits will be reached, when the number of users and requests increase.

👍
For up-to-date Object Storage technical specifications, see Quotas and limits.

Use CDN offloading

You can overcome this problem by using the Akamai CDN which caches and delivers content closer to your end users. Once the CDN caches an object like a song, video, or image, it doesn't need to be pulled from origin storage again until the object file changes or the cache timeout expires. For example, by offloading requests to the CDN, a single MP3 audio file downloaded 1 million times only requires a single request to the source bucket.

Use a distributed-bucket architecture

The Akamai CDN can definitely help get around some limitations of a single-bucket architecture. You can also improve and optimize your Object Storage performance further by distributing content across multiple buckets. This helps with:

Overall load distribution
Increased requests-per-second (RPS) capacity
Segmentation of content across endpoints can act as a security measure in case of compromise
Distribution of the number of single endpoint requests from the CDN

Rather than placing music, video, and image content folders in the same top-level directory, you could triple RPS capacity by placing each type of content in their own buckets: store-music, store-video, and store-images. In the example below, additional content is placed underneath the top-level directory of each bucket:

Paths for the store-music bucket include: store-music, store-music/copyright, and store-music/free.
Paths for the store-video bucket include: store-video.
Paths for the store-images bucket include: store-images.

As the site scales, you can create additional buckets and move sub-category content, such as copyrighted or free music, to their own buckets. This increases RPS capacity by further distributing the number of endpoints the CDN can access to cache content. In this example, copyrighted music is stored in a new copyright bucket. RPS capacity is now four times the original bucket architecture:

Paths for the copyright bucket include: copyright.
Paths for the store-music bucket now include: store-music and store-music/free.
Paths for the store-video bucket include: store-video.
Paths for the store-images bucket include: store-images.

CDN considerations

Each bucket in your architecture has the ability to serve as a single origin endpoint from which the Akamai CDN can pull content. This results in a distributed backend with less opportunities for RPS limits to be reached. The more distributed your backend architecture is, the higher the capacity for scaling.

Relationship to bucket design

CDNs can often overcome flaws of poorly architected environments. However, when a bucket architecture is designed well, the benefits can directly translate to the CDN. Object Storage bucket architecture should be designed to be functional, scalable, performant, resilient, and cost efficient so that the Akamai CDN serves your content as effectively as possible.