Download with NetStorage

NetStorage's primary purpose is a delivery origin, not general-purpose storage. Downloading content can actually refer to two separate operations. You should have a good understanding of the difference between the two.

ūüöß

NetStorage isn't designed for use as a ‚Äúdrop box‚ÄĚ or direct downloads. Content management APIs and session protocols aren't intended for bulk downloads or to deliver content to end users.


Use NetStorage as an origin

Create your NetStorage environment

Before you can use NetStorage as your origin, you need to it set up.

  1. Create a storage group. This is where you'll upload and store your website or app content for delivery with the delivery product you're using with your property. Gather some values during this process, to help you pick the storage group that'll serve as your origin:

    • The Storage Group Name. You'll set up a unique name for your storage group.

    • The CP code assigned to the Upload Directory. An upload directory is the root directory in your storage group. When setting it up, you'll assign a numeric content provider (CP) code to the upload directory, for use in billing and reporting.

  2. Create an upload account. You use an upload account to access your storage group to upload content to it. You can set up various access methods based on popular protocols such as SFTP, SecureCopy, rsync, and Aspera.

  3. Upload your site or app content to NetStorage. Use any of your upload account's access methods to add the content to your storage group.

Add NetStorage as the origin for your property

Next, you need to create or configure your property. During this phase, you'll define property hostnames and any optional property variables. Be aware that:

  • The selected delivery product's pricing applies.
  • Property Manager will require the use of "Tiered Distribution" with NetStorage origins, even though the extra caching tier is usually not useful to increase offload when the other CDN is functioning well. But like the following minimum TTL, it is mandated to protect ‚ÄčAkamai‚Äč from sudden traffic spikes. Also, note that Tiered Distribution increases the billable intra-‚ÄčAkamai‚Äčmidgress transfers.
    • The required Property Manager configuration has a minimum Time to Live (TTL) setting of 10 minutes, and it offers limited security options. (Security options include HTTPS delivery with a shared TLS certificate, IP Access Control Lists, and fixed HTTP header verification.)
    • Time-expiring authentication tokens are recommended and require advanced metadata. This advanced metadata is not currently self service, and you need assistance from ‚ÄčAkamai‚Äč Professional Services to implement it.

These workflows are available based on your delivery product:

Property type

Workflow

Adaptive Media Delivery (AMD)

NetStorage origin prerequisites for Adaptive Media Deliver (AMD).

Download Delivery (DD)

NetStorage origin prerequisites Download Delivery.

ION

The NetStorage origin server workflow for ION.

Property Manager

The NetStorage origin server workflow for Property Manager.


Use NetStorage as a queuing service

Do this using Log Delivery Service (LDS). These limits apply:

  • Standard write operation rate limits apply, see technical limits with NetStorage for more information.
  • You can use NetStorage as a place to queue up logs before taking them to your own system. This includes components such as Log Delivery Service (LDS). However, ‚ÄčAkamai‚Äč doesn't support using NetStorage to write files once and then download them using the CMShell download protocols/APIs with no delivery activity.

Downloads to deliver content to end users

NetStorage is designed to store content that you then deliver to your end users via any of our associated content delivery products. End users obtain content through an EdgeServer, and in the case of a cache miss, the EdgeServer will retrieve the content directly from NetStorage.

ūüďė

‚ÄúCache hit‚ÄĚ and ‚Äúcache miss‚ÄĚ refer to whether or not EdgeServers hold the requested content in their cache.

Building the delivery URL

The Delivery URL is used to access the actual content. This is the URL you provide to your end users for access, based on the ‚Äúdigital property‚ÄĚ defined in your account‚Äôs specific Edge configuration. The construction of this actual URL can vary, based on your selected delivery method (for example, our Adaptive Media Delivery or Download Delivery solutions).

Example: Content delivery to a browser

Once you have uploaded content to NetStorage and properly established delivery URLs into your web site, end-user browsers can request NetStorage content from EdgeServers.

Example delivery flow

  1. An end user requests content from the target web site via a properly formatted delivery URL, and the intelligent DNS system directs the browser to the optimal EdgeServer. In the event of a cache hit, the EdgeServer will simply return the content to the browser. In the event of a cache miss, the EdgeServer must retrieve the content from NetStorage.
  2. The Mapping System is used:
    1. A DNS Request for the appropriate Storage URL domain name is sent.
    2. The Storage URL domain name is resolved to the optimal NetStorage site based on real-time Internet traffic mapping.
  3. Interaction between the EdgeServer and NetStorage site:
    1. The EdgeServer processes the request from the Mapping System and requests content from the optimal NetStorage Site.
    2. Content from the NetStorage site is delivered to the requesting EdgeServer.
  4. The EdgeServer delivers the content to the browser and stores it in cache for future requests.

Content management downloads are intended for data recovery

Use content management protocols for data recovery when the original source is lost.

Content management downloads don't provide the same level of reliability as with CDN delivery products. CMS connections lack the additional resources available to edge servers. Be aware of the following when using NetStorage with the CMS API and session-based protocols for downloads:

  • Checks are not made to find recently uploaded content at another replica location. Content at that replica site may not have replicated to your current location.
  • The CMS API typically trusts the end-to-end MD5s provided with any download request to quickly validate the file transfers. It's not intended to protect against files modified at the source.
  • Downloads larger than 2 GB. When downloading files in excess of 2 GB, be prepared to retry the request if the TCP connection fails. Large file transfers can experience the following:
    • Long connections are more likely to fail in the middle and must restart from the beginning of the file.
    • Extra bandwidth behind the scenes when a retry attempt is requested from the start of the file.
  • Large quantities of files may affect your downloads. While there is no set maximum on the number of files that can be downloaded, it is important to note that content management protocols aren't intended for use as a ‚Äúmass download‚ÄĚ resource. If used to download a large number of files, you may notice performance issues and possible failures.

ūüďė

If you intend to download a large number of files it's recommended that you incorporate a delivery configuration with "Large File Optimization".


Unsupported NetStorage downloads

Avoid using unsupported NetStorage download workflows.

  • NetStorage as a ‚Äúdrop box‚ÄĚ. Don't use CMS APIs to store-and-forward or replicate content to distant partners or business-to-business (B2B) apps. This is not a suitable use case.

These protocols are for content management purposes. They're not intended for bulk download or to deliver content to end users. Avoid using them for this purpose.

  • Aspera Upload Acceleration, FTP
  • FileManager
  • NetStorage Usage API
  • SFTP
  • Secure Copy (SCP)
  • SSH
  • Rsync

Did this page help you?