Use tags and prefixes in lifecycle policies to delete objects
Lifecycle policies let you automate content management in your buckets. You can use a lifecycle policy to automate simple tasks, such as deleting all objects in a bucket after X days. More complex filters are available to help you identify objects based on the rules that you define. Tags and prefixes provide additional flexibility when managing buckets with objects that have different lifecycles.
About Tags
You can use a lifecycle policy filter to only delete objects with a specific tag after X days. This allows your application to declare the lifecycle of the object by automatically deleting tagged objects when the object reaches the age you define.
For example, you can create the following tags, 1day
, 10day
, and 30day
and apply them to delete objects 1, 10, and 30 days after they were created.
About Prefixes
You can also use a lifecycle policy filter to delete objects with a specific prefix after X days. With this approach you can define a prefix to limit the deletion of objects based on a prefix for the key (name) of the object.
In this example, customer objects within a bucket are created with the prefix as a part of their key that includes a customer id. Each customer requires their own specific object lifecycle. Here are some example object keys:
customerid1/object1.txt
customerid1/object2.txt
customerid2/object1.txt
customerid2/object2.txt
With this key naming convention the lifecycle policy for the bucket can ensure that objects prefixed with customerid1/
have a lifecycle of 30 days. Objects prefixed with customerid2/
have a lifecycle of 60 days.
Define a lifecycle policy with a tag filter
This example shows you how to add a tag to a lifecycle policy to delete a tagged object from a bucket after one day.
- To use a lifecycle policy to delete objects based on tags, create a policy xml file.
<LifecycleConfiguration>
<Rule>
<ID>delete-tagged-objects</ID>
<Filter>
<Tag>
<Key>retention</Key>
<Value>1day</Value>
</Tag>
</Filter>
<Status>Enabled</Status>
<Expiration>
<Days>1</Days>
</Expiration>
</Rule>
</LifecycleConfiguration>
- Set the policy for the bucket.
$ s3cmd setlifecycle delete_all_after_1day_wtags.xml s3://example-bucket
s3://example-bucket/: Lifecycle Policy updated
- Check the policy for the bucket.
$ s3cmd getlifecycle s3://example-bucket
<?xml version="1.0" ?>
<LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<Rule>
<ID>delete-tagged-objects</ID>
<Filter>
<Tag>
<Key>retention</Key>
<Value>1day</Value>
</Tag>
</Filter>
<Status>Enabled</Status>
<Expiration>
<Days>1</Days>
</Expiration>
</Rule>
</LifecycleConfiguration>
- Set the tag on the object.
$ s3cmd settagging s3://example-bucket/test_file.txt retention=1day
s3://example-bucket/test_20241128.txt: Tagging updated
s3cmd gettagging s3://example-bucket/test_20241128.txt
s3://example-bucket/test_20241128.txt (object):
retention: 1day
Any objects with the tag will be deleted after one day.
retention: 1day
Define a lifecycle policy with a prefix filter
This example shows you how to add a prefix filter to a lifecycle policy that will delete an object from a bucket if it includes the prefix “abc”.
- To use a lifecycle policy to delete objects based on an object name prefix, create a policy xml file.
<LifecycleConfiguration>
<Rule>
<ID>delete-prefixed-objects</ID>
<Filter>
<Prefix>abc</Prefix>
</Filter>
<Status>Enabled</Status>
<Expiration>
<Days>1</Days>
</Expiration>
</Rule>
</LifecycleConfiguration>
- Set the policy for the bucket.
$ s3cmd setlifecycle delete_all_after_1day_wprefix.xml s3://example-bucket
s3://example-bucket/: Lifecycle Policy updated
- Check the policy for the bucket.
$ s3cmd getlifecycle s3://example-bucket
<?xml version="1.0" ?>
<LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<Rule>
<ID>delete-tagged-objects</ID>
<Filter>
<Prefix>abc</Prefix>
</Filter>
<Status>Enabled</Status>
<Expiration>
<Days>1</Days>
</Expiration>
</Rule>
</LifecycleConfiguration>
- Create an object named "abcde/fg.txt".
$ touch empty-file
$ s3cmd put empty-file s3://example-bucket/abcde/fg.txt
upload: 'empty-file' -> 's3://example-bucket/abcde/fg.txt' [1 of 1]
0 of 0 0% in 0s 0.00 B/s done
Any objects that include the prefix "abc" will be deleted after one day.
Limits
The following limits apply when using tags to apply a lifecycle policy.
Description | Limit |
---|---|
Number of tags that can be set at a time per object | Maximum of 10 tags. |
Rate limit for setting Object Storage tags | For E2/E3 rate limits, GetObjectTagging counts as a GET and PutObjectTagging counts as a PUT . |
Maximum file size for the number of tags supported by the API | Tag keys are limited to 128 Unicode characters each, tag values are 256 characters. Every Unicode character counts as one character against the 128 or 256 character limit. There is no relationship or restriction between tags and object size. |
Updated about 10 hours ago