Use tags and prefixes in lifecycle policies to delete objects

Lifecycle policies let you automate content management in your buckets. You can use a lifecycle policy to automate simple tasks, such as deleting all objects in a bucket after X days. More complex filters are available to help you identify objects based on the rules that you define. Tags and prefixes provide additional flexibility when managing buckets with objects that have different lifecycles.

About Tags

You can use a lifecycle policy filter to only delete objects with a specific tag after X days. This allows your application to declare the lifecycle of the object by automatically deleting tagged objects when the object reaches the age you define.

For example, you can create the following tags, 1day, 10day, and 30day and apply them to delete objects 1, 10, and 30 days after they were created.

About Prefixes

You can also use a lifecycle policy filter to delete objects with a specific prefix after X days. With this approach you can define a prefix to limit the deletion of objects based on a prefix for the key (name) of the object.

In this example, customer objects within a bucket are created with the prefix as a part of their key that includes a customer id. Each customer requires their own specific object lifecycle. Here are some example object keys:

customerid1/object1.txt
customerid1/object2.txt
customerid2/object1.txt
customerid2/object2.txt

With this key naming convention the lifecycle policy for the bucket can ensure that objects prefixed with customerid1/ have a lifecycle of 30 days. Objects prefixed with customerid2/ have a lifecycle of 60 days.

Define a lifecycle policy with a tag filter

This example shows you how to add a tag to a lifecycle policy to delete a tagged object from a bucket after one day.

  1. To use a lifecycle policy to delete objects based on tags, create a policy xml file.
<LifecycleConfiguration>
   <Rule>
       <ID>delete-tagged-objects</ID>
       <Filter>
           <Tag>
               <Key>retention</Key>
               <Value>1day</Value>
           </Tag>
       </Filter>
       <Status>Enabled</Status>
       <Expiration>
           <Days>1</Days>
       </Expiration>
   </Rule>
</LifecycleConfiguration>

  1. Set the policy for the bucket.
$ s3cmd setlifecycle delete_all_after_1day_wtags.xml  s3://example-bucket 
s3://example-bucket/: Lifecycle Policy updated
  1. Check the policy for the bucket.
$ s3cmd getlifecycle s3://example-bucket                    
<?xml version="1.0" ?>
<LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
	<Rule>
		<ID>delete-tagged-objects</ID>
		<Filter>
			<Tag>
				<Key>retention</Key>
				<Value>1day</Value>
			</Tag>
		</Filter>
		<Status>Enabled</Status>
		<Expiration>
			<Days>1</Days>
		</Expiration>
	</Rule>
</LifecycleConfiguration>

  1. Set the tag on the object.
$ s3cmd settagging s3://example-bucket/test_file.txt retention=1day
s3://example-bucket/test_20241128.txt: Tagging updated
s3cmd gettagging s3://example-bucket/test_20241128.txt
s3://example-bucket/test_20241128.txt (object):
	retention:	1day

Any objects with the tag will be deleted after one day.

retention:	1day

Define a lifecycle policy with a prefix filter

This example shows you how to add a prefix filter to a lifecycle policy that will delete an object from a bucket if it includes the prefix “abc”.

  1. To use a lifecycle policy to delete objects based on an object name prefix, create a policy xml file.
<LifecycleConfiguration>
   <Rule>
       <ID>delete-prefixed-objects</ID>
       <Filter>
           <Prefix>abc</Prefix>
       </Filter>
       <Status>Enabled</Status>
       <Expiration>
           <Days>1</Days>
       </Expiration>
   </Rule>
</LifecycleConfiguration>

  1. Set the policy for the bucket.
$ s3cmd setlifecycle delete_all_after_1day_wprefix.xml  s3://example-bucket  
s3://example-bucket/: Lifecycle Policy updated
  1. Check the policy for the bucket.
$ s3cmd getlifecycle s3://example-bucket                       
<?xml version="1.0" ?>
<LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
	<Rule>
		<ID>delete-tagged-objects</ID>
		<Filter>
			<Prefix>abc</Prefix>
		</Filter>
		<Status>Enabled</Status>
		<Expiration>
			<Days>1</Days>
		</Expiration>
	</Rule>
</LifecycleConfiguration>
  1. Create an object named "abcde/fg.txt".
$ touch empty-file  
$ s3cmd put empty-file s3://example-bucket/abcde/fg.txt  
upload: 'empty-file' -> 's3://example-bucket/abcde/fg.txt'  [1 of 1]  
 0 of 0     0% in    0s     0.00 B/s  done

Any objects that include the prefix "abc" will be deleted after one day.

Limits

The following limits apply when using tags to apply a lifecycle policy.

DescriptionLimit
Number of tags that can be set at a time per objectMaximum of 10 tags.
Rate limit for setting Object Storage tagsFor E2/E3 rate limits, GetObjectTagging counts as a GET and PutObjectTagging counts as a PUT.
Maximum file size for the number of tags supported by the APITag keys are limited to 128 Unicode characters each, tag values are 256 characters. Every Unicode character counts as one character against the 128 or 256 character limit. There is no relationship or restriction between tags and object size.