Real-time content

Real-time describes content that is subject to a "real-time constraint," such as news broadcasts, APIs and live streaming events. All content uploaded to NetStorage is subject to the Eventual Consistency Model, and it may impact the regular, timely updates that you require for this form of content.

What's considered real-time content

The points that follow address some of these use cases, to serve as a warning of potential issues that may occur with NetStorage. They have typically seen the highest failure rates on NetStorage because they exceed the recommended rate limits described.

The HTML pages associated with sporting events, election information, or other news
The manifest files for live streaming
The use of NetStorage as an API
Real-time NetStorage objects: This could be a fixed URL to the latest news, sports, weather, emergency, HTML page or image, or any other frequent overwriting of existing objects to simulate dynamic content.

Associated Delivery Products and Time to Live (TTL) Settings

By default, the lowest delivery TTL you can directly set for an associated delivery product (via Property Manager) is 10 minutes, when you are using NetStorage as your origin. We may allow this to go as low as 1 minute with the appropriate business case and controls like limited URLs and an understanding of the intended traffic load. Contact your account representative and share your business case to determine what setting can be applied for your instance.

Re-uploading objects will impact TTL

Note that even low TTLs will not ensure overwritten objects will always be delivered to requesting end users within the TTL after they are uploaded. When re-uploading objects, the older versions of the content may be re-cached by the CDN when replication of the newer version to the other NetStorage replica is delayed. That delay is usually less than a second if no other change was also pending on the same CP code, but it is an indeterminate and possibly lengthy time. These replication delay issues tend to occur more often when the same object is re-uploaded more frequently.

📘
Objects uploaded to never-before-used URLs do not suffer from this issue.

Images and other small objects

This describes objects with an average file size that's less than one megabyte (1 MB). Using 2MB or larger objects is recommend to reduce the impact of NetStorage limits and increase upload performance.

Note: Symlinks and POSIX-compatible directories are considered objects (and count toward this use case). Individual files within a Zip file are not.

Limits that apply to this use case

Standard read and write operation rate limits apply.
Total object count limits per storage group apply.

Directories with many objects

Be aware that Technical limits with NetStorage apply. If you want to perform "dir" operations, performance may suffer for a single directory that contains a large amount of objects. Depending on the protocol, eventually client timeouts could occur.

NetStorage for Live Streaming Origin

Best practice: Don't use NetStorage

We recommend that you implement our Media Services Live Origin (MSLO) component instead. Contact your Account Representative for additional information and details on obtaining this product.

What if I choose to use NetStorage for this use case?

If you decide against using MSLO and wish to use NetStorage instead, you need to be aware of the following limitations:

Rapidly updating manifests for this format of content may see higher failure rates for delivery to NetStorage. (This is because their delivery requires rates above the recommended limit.)
Points addressed in the Images and other small objects use case also apply here, if either of the following apply to the content:
- Very low latency is required: Content segments are less than two seconds in length.
- The content uses a low bitrate: This includes single content streams that are less than one Mbps, many content streams that are less than one Mbps, or any audio-only streams.
If you have "lots of streams" (for example, you're incorporating Live Streaming as a service, and a lot of objects are being delivered to NetStorage), the Images and Small Objects use case also applies.
Standard write operation limits apply.

NetStorage and General Purpose Compute

NetStorage is not a true general purpose compute (GPC) solution. It's designed to serve as an origin for the delivery content. If you're looking to NetStorage to serve as a true GPC, consider the following:

While POSIX directory semantics can be provided, you cannot append or update partial objects with NetStorage.
We provide some application-specific compute services that can write to NetStorage. This includes various session-based upload protocols such as Rsync, as well as access via our Video on Demand Transcoding service, but not GPC. NetStorage has no special integration with other cloud vendor compute options that would reduce either of the following:
- Data Transfer Costs: There are no direct connections.
- Latency: You cannot select a "near-other-vendor" location.
Standard write rate limits apply.

Security-specific use cases

This describes what NetStorage supports, and does not support for specific security use cases.

What's supported

NetStorage is FedRAMP-compliant (to support the FedRAMP Security Assessment Framework: https://www.fedramp.gov/)
NetStorage provides many other features to prevent tampering and the corruption of data.
- You can include an MD5 on upload. MD5s are checked in every transfer in the replication and delivery chain. They are also periodically checked at-rest.
- The use of standard, non-secure FTP is discouraged (SFTP is supported), but if FTP is absolutely needed, support is provided for IP Address/Geographic Region Access Control Lists, recent activity audits and automated password rotation. Set up of these options is performed on a per-upload account basis, and they are fully discussed in the NetStorage Configuration documentation.

What's not supported

PCI compliance and encrypt-at-rest features to absolutely prevent disclosure of content from those with access to the hardware are currently not supported.
Integration with third-party authentication such as OAuth/LDAP is currently not supported.

Workarounds

If you need support for encrypt-at-rest security use cases, we recommend that you encrypt your content before uploading to NetStorage, and after downloading.

We allow many users with different permissions, and we offer APIs to control this, but we currently have no user management integration with OAuth/LDAP. See an alphabetical list of Akamai products and tools for information on which one fits your workflow.

Country-specific location guarantees

Country-specific location guarantees are often needed by health care and government data, but NetStorage doesn’t fully support all countries and all applications.

As a workaround, you can apply application-level encryption to your content and keep the associated keys in-country.

NetStorage as a cache

This use case assumes you are storing popular or recent content in NetStorage, and you've set up your Content Delivery Network (CDN) to try another origin when it can't immediately find this content in NetStorage.

Limits can apply to this use case

This use case may result in "extreme usage," and negatively affect your performance. Extreme usage applies if you approach or exceed standard read and write operation rate limits.
The offload hit rate NetStorage provides should be high (at or above 90%) because the misses will be slow. Offload hit rate describes the percentage of successfully resolved requests for content. A response for content not found from NetStorage (a "miss" or 404) is slow because all of your replicas are checked before returning the response. This requires extra processing, and it can impact your overall NetStorage performance.
NetStorage doesn’t provide a "last accessed" timestamp for objects. This forces you to manually track the objects that should be included in the cache. Take caution to avoid inadvertently deleting relevant content. (Remember that the available timestamp represents its "create" date, not its last accessed date.)
It's often better to fine-tune the CDN to perform better caching instead of using NetStorage. (NetStorage is designed to be a highly durable storage system, and not particularly as a caching destination.) If you do, you'll avoid incurring the extra costs that the necessary, redundant copies can imply.