How Exactly Does Amazon S3 Object Expiration Work?
I am always intrigued by Amazon’s S3
as it is a great way to store for short and long term and how they keep adding new features.
Amazon Simple Storage Service
(Amazon S3) is a cloud computing web service offered by AWS (Amazon Web Services) provides object storage which helps to store and retrieve any amount of data from websites and mobile apps, corporate applications, and data from IoT sensors or devices.
It is designed to deliver 99.99% durability, and stores data for millions of applications used by market leaders in every industry.
There is a very good blog post
from Jeff Barr that details out Object Expiration Policy. But I came across a very simple question that puzzled me and it took some time to find an answer to it, so I thought I should post this question which is given below.
There was a similar question asked on Stack Overflow
which basically says what time does transition policy apply to an object? Will the object gets transitioned at Midnight or it will get transitioned after 24 hours?
For e.g.: Suppose I mark an object for deletion on 13th at 5:30 PM
so when will the object be deleted? By next day i.e. 14th at 5:30 PM
or will it be deleted on 15th at Midnight
As per AWS Documentation:
We calculate the expiration date for an object by adding that object’s creation time to the expiration period and rounding off the resulting time to midnight of that day.
AWS evaluates the expiration rules once in a day. During the evaluation process if any object found to be expired as per the expiry date then that object will be queued for removal. You will not be billed for any associated storage for those objects on or after their expiration date. If server access logging has been enabled on S3 bucket, then S3.EXPIRE.OBJECT
record will be generated when the object gets removed.
To simplify this based on the above example, AWS Calculates everything based on UTC. To explain this scenario, object was marked for deletion on 13th 17:30
UTC after a day, AWS will check at Midnight 14th
i.e UTC 00:00
but the criteria for One Day would not meet till then and this object will not be deleted, so by next day i.e 15th at midnight00:00 UTC
, object will get checked again and this time condition will be matched and object will be deleted on 15th at midnight.
Hope this clears out one of the important topics and helps us better understand how Object Life cycle Policy Works.