NetStorage has active rate-limiting in place to recognize unreasonable bursts. This is not solely determined by your specific usage.
This API doesn't redirect ('chase') requests to other NetStorage replicas. This is especially important if your NetStorage configuration uses replica-specific hostnames, because you may be interacting with content that's outdated or missing. Interacting with outdated content may occur in these situations:
- If your content isn't in the initial replica region.
- If the content you're accessing isn't the most recent version.
We recommend using a delivery product, as those requests will redirect across all NetStorage replicas of the target storage group. This ensures delivery of the freshest content even if the first delivery request goes to a replica which doesn’t yet have the data.
Network availability is subject to unreasonable bursts or temporary overloads. This is likely to affect you if your usage is contributing to the problem, or if other customers are performing similar actions to the shared resource at the same time.
NetStorage supports both short- and long-term models:
- Short term. Via DNS load-balancing, we try to spread reasonable bursts to fit the capacity of online locations that are holding relevant content.
- Long term. We use automated content movement to re-balance content locations.
View throttled operations in a Read & Write Operations Report. Use this report to determine what types of operations are exceeding the rate limit thresholds. Contact your account team to adjust your threshold limits.
NetStorage doesn't support a "fixed" number of operations per second. Multiple factors can impact this total, so we don't guarantee a static number of operations per second, per storage group. Instead, these are the recommended “best practice” maximums:
Type of limit
Storage group size
Number of objects
Objects per directory
Client timeouts may occur if a directory list contains an excessive amount of objects.
Maximum CP codes
Write operations per second
Read operations per second
Maximum upload accounts
90 concurrent connections per upload account.
Creating or updating a storage group configuration
10 operations per hour.
Getting storage group configuration details
50 operations per hour.
File path length
800 characters per object.
150 GB single object maximum.
Naming of files and directory paths
Type of limit
Maximum archive size
4 GB archive size.
Maximum number of files in a “zip” archive
65,000 files per archive.
Capabilities beyond the basic UNIX "unzip" are not supported, such as:
You should try Serve from Zip
You might be able to serve your files from within an uploaded zip archive on NetStorage without unzipping it, by using Serve from Zip. Serve from Zip supports much larger files and the zip64 file format. If you can use it, you won't need to use this unzip command. See if the Serve from Zip considerations are compatible with your workflow.
Type of limit
Maximum file size
Maximum number of files in an archive
When using the non-HTTP content management protocols (Aspera, rsync, ftp, scp, sftp, cmshell) their actions are converted into corresponding HTTP-API requests by Akamai.
Origin for content delivery
There is a read request for every attempted object access. These are often the cause of most read requests.
Requests counted include those with these HTTP response codes:
Successful, including 204 "Partial Content".
Redirections, including 304 "Not Modified".
Client errors, including 404 "Not Found" (and 429 "Rate Limited"?), but excluding some errors which we can't attribute to the right customer. E.g., some 400s, 401s, and 403s.
If you require more than 600 total storage CP codes on your contract, establish different storage groups to use different geographic locations. A single storage group cannot span multiple geographic regions.
Set up individual upload accounts to access a specific CP code (upload directory), and limit directory access via Subdirectory Restrictions. This way, the individual users associated with a specific upload account will only be able to access that directory. You can then use the NetStorage HTTP API or the CMShell (via the "du" command) to report the total footprint for the directory. While it offers a method of tracking, it doesn't offer the robust reporting and graphs that you have access to via Control Center Reports and Analytics that are driven by CP codes.
Typically, you can get by with a just a few total upload accounts. (For example, ten accounts, plus one more for each additional storage group.)
This page provides situational information, and some possible suggestions to accommodate those situations. Also see how User login limits are enforced for specific recommendations on avoiding login failures for non-NetStorage HTTP API upload protocols.
Upload account needs
If you need more than 100 upload accounts
If you need more than 10 concurrent connections, and are uploading large files in each login session
Note: The average sustained change rate across the storage group still matters, regardless of the upload protocol you're using.
If you have more than 1,000 TB stored in one storage group
Look into "Sharding" content to multiple storage groups.
If you have more than 200 million objects in one storage group
If you are working with files larger than one (1) GB in size
If you are working with files larger than 100 GB in size
High levels of read/write requests ("bursts") or temporary overloads will result in rate limits, for example, delays in responses. If the clients are not moderated by this (e.g., there are many clients acting in parallel) then further requests may be denied. (For example, a 429 response code is sent on HTTP protocols.)
There is no guaranteed level before these rate limits kick in, but they are applied first to storage groups that are most likely causing the overload. So, we recommend that you stay under the "suggested requests per second" values. When evaluating your workflow, the following apply:
About these rates
These numbers consider all requests to single storage group, even if one replica is distant or unavailable. On average, you are likely to see higher upload and download request rates to a storage group. However, on a single TCP connection you should expect lower request rates. For example, including common network delays, 10 write operations per second on a single connection is normal for many customers uploading small files.
These rates also assume that requesting clients are adhering to the normal DNS best practices for use regarding TTL settings, and assume that applications are not "sticking" to the IP addresses returned in the past. Otherwise, you reduce the chances we have to map your requesting clients away from overloaded servers.
Sending more than 1,000 read requests per second to a single storage group
Sending more than 50 write requests per second (sustained rates averaged over 10 seconds) to a storage group
Your replication configuration will impact rate limits
Note: The following configurations require additional configuration before they are enabled. They are not available by default due to potential performance issues:
Concurrent login sessions use shared resources. We enforce fairness policies to limit the impact that each customer, upload user account and storage group may have on each other. Increasing the number of concurrent logins for one customer, one upload user account and one storage group raise the probability that a login denial will occur. Note these recommendations:
- Large File Transfers: In this case, rate of progress is limited by data transfer throughput. More connections via additional Upload User Accounts are unlikely to improve things.
- Small File Transfers: In this case, rate of progress is limited by turnaround time to get responses. Additional Upload User Account connections can be seen as a reasonable way to improve things.
What can I do if I get a login denial?
You can attempt a retry strategy that involves the following:
- Try re-resolving the Upload Domain Name and creating a new network connection on every login attempt. This allows your application to avoid overloaded upload servers, by choosing alternate IP address answers from your client’s past cached DNS responses; or new IP addresses as we identify overloaded servers and the DNS time to live (TTL) expires.
- Implement an increasing "back-off time" to ensure your application is not the cause of the overload. For example, back-off your attempts at increasing intervals, such as zero, five, 10 and 20 seconds apart, and then attempt every 30 seconds.
Be aware of the design implementation constraints of NetStorage.
NetStorage provides complete redundancy with wide geographical separations, as well as redundancy within each data center. This allows high performance and availability for uploads and downloads, because requests are DNS load-balanced to the best location for a client. This gives you high durability for your content, even in the face of natural disasters of regional scope. To maintain this redundancy, geographical replication is performed after an upload or failure, and the last request from the customer to change the same object eventually wins.
NetStorage is optimized for large media object delivery workloads to clients across the world. (For example, this includes tools that deliver video on demand media or perform software downloads.) Additionally, NetStorage can be used for other high availability and durability applications that fit its system limits, and can also cope with eventual consistency.
These use cases are problematic and require a more in-depth understanding of how NetStorage works:
- You have a large total number of GB to store.
- You have a large number of files to store.
- You work with very large files.
- You deal with files that change frequently.
- You require high bandwidth uploads.
- Your files are uploaded from large numbers of geographically diverse sources.
- You need to avoid delivering stale content after a change.
- You have many delivery requests for large numbers of files that have not yet been uploaded. (These return a 404 error.)
With each of the above use cases, there are specific aspects you need to keep in mind.
- System Limits
- Rate Limits
- Upload Account and User Connection Limits
- Geographic Replication and the "Eventual Consistency Model"
However, symbolic links ("symlinks") are fully supported.
Control of all scratch directory parameters and behaviors are retained internally by Akamai.
File Manager provides a basic UI that you can use to browse and manage content stored in NetStorage via Control Center. It is meant to be used for small and quick operations.
You should not use File Manager and instead, implement a production file upload system, if any of the following apply:
- You need to preserve directory structures.
- You need to traverse through complex directory trees.
- You need to initiate large data transfers, or upload large files.
- You need to perform other "mass operations," such a deleting or moving a large number of files.
File Manager does not preserve nested sub-directory structures on upload, and nested files will reside in the top-level directory.
A "production file upload system" is one you develop using the various protocols available with NetStorage, or the NetStorage Usage API.
Because of replication and certain other features involved in NetStorage, you cannot rename directories. Some workarounds include the following:
- You can create symbolic links between directories. You can then use a desired, new name for the symbolic link.
- You can "rewrite" a path via an associated media delivery product. You can set up the configuration for an associated delivery product to rewrite the directory path. For example, using Adaptive Media Delivery you can set this up using the Modify Outgoing Request Path behavior in Property Manager.
With this in mind, take care when naming your directories in a storage group. Ensure the name you've chosen will fit your needs in the future.
A limited number of file attributes (for example, setting up "read only" status) can be set up in an upload account using the NetStorage Groups UI in Control Center. However, most file attributes such as read, write, ownership and execute (attributes other than the time stamp) are not self-service. Speak with your account representative for details on setting these attributes.
Updated about 1 month ago