Application design best practices

Consider the contents of these sections if you're using Object Storage to serve a high-throughput application.

Scale the number of TCP connections

A fundamental design pattern for high-throughput applications is to distribute content over multiple TCP connections. Take this into consideration, whenever you access Object Storage. For client applications, multiple TCP connections across multiple threads can help increase overall performance by minimizing the impact of blocking API calls, and better utilizing CPU resources. Similarly, multiple TCP connections also allow for multiple network paths to be followed, which optimizes throughput across the network.

Distribute requests across multiple IP addresses

Each DNS lookup for an Object Storage S3 hostname, like us-sea-9.linodeobjects.com randomly returns 12 IP addresses from a larger pool of addresses. Each DNS A record's time to live (TTL) has a timeout of 30 seconds.

• You should use a diversity of IP addresses when connecting to Object Storage to help ensure optimal throughput rates. This distributes content across multiple ingress points to the Object Storage service.
• You need to ensure that applications respect the TTL of the A records, and resolve again when the TTL expires.
• You should review any local libraries, software development kits (SDKs), or caches to ensure that connection requests are spread across all of the IP addresses available for each of your Object Storage endpoints.
• Your applications should model for a maximum throughput of 1 Gbps, per connection. By spreading uploads across multiple connections with a model target of 1 Gbps each, you can reach a higher overall throughput rate. For example, an application may issue eight simultaneous GET requests, to eight distinct IP addresses to target a download rate of eight Gbps.

Throughput rate limiting

If request rates exceed your per endpoint quotas, individual connections can be throttled. For uploads, throttling results in TCP back-pressure on the sender, which limits the amount of data that's transmitted to either the per connection or per endpoint throughput quota.

Use larger objects

Larger objects that are greater than 1 MiB are preferred over smaller objects, for high-throughput applications. Larger objects minimize the TCP connection and operation overhead with individual S3 API requests. This can positively impact your overall application throughput, for large numbers of small objects.

Examples when using Amazon AWS S3 SDKs

Increase connection pool size

Most Amazon AWS S3 SDKs default to a small connection pool size. For instance, the Java SDK defaults to 50, while Python/boto3 defaults to 10. At high throughput, most requests funnel into a handful of persistent connections, that potentially go to the same IP address. Here are some example SDK configurations to increase the connection pool size:

• Java SDK v2:
  Java

  ClientOverrideConfiguration.builder().httpClientBuilder(ApacheHttpClient.builder().maxConnections(200))

• Golang SDK: Tune MaxIdleConns and MaxIdleConnsPerHost in the underlying http.Transport
• Python/boto3:
  Python

  botocore.config.Config(max_pool_connections=100)

Increase worker or thread count

For serial uploads and downloads, connection optimizations or local caching of DNS addresses, can reuse the same TCP connection or destination IP address. If this is an issue, you can use parallel workers or threads in your application to spread requests across multiple connections and IP addresses. You can also do this by running multiple instances of your client.

Adjust retry behavior on failures

While retries or failovers are essential, you should make sure they don’t all target the same IP address. As a best practice, implement exponential backoff with jitter, using standard mode in the SDK retry configuration. Here are some example configurations using Amazon AWS S3 SDK’s:

• Java SDK v2:
  Java

  RetryPolicy.builder().numRetries(3).backoffStrategy(BackoffStrategy.defaultStrategy())

• Python/boto3:
  Python

  botocore.config.Config(retries={'max_attempts': 5, 'mode': 'standard'})

In most cases, you should use the default retry behavior of standard mode. Adaptive retry is an option, if your application has a particular sensitivity to failed requests and can accept slightly higher latency as a trade-off.