Lifecycle Policy For AWS S3 Bucket

Why We need Lifecycle Policy For S3 Bucket ?

Implementing Lifecycle policies within S3 is a great way of ensuring your data is managed safely (without experiencing unnecessary costs) and that your data is cleanly deleted once it is no longer required. Lifecycle policies allow you to automatically review objects within your S3 Buckets have the objects deleted from S3.

You may want to do this for Cost Saving, legislative compliance, internal policy compliance, or general housekeeping.

Implementing good lifecycle policies will help you increase your data security. Good lifecycle policies can ensure that sensitive information is not retained for periods longer than necessary.

Why and Where We Can Use this Policies of Lifecycle in our Organization?

We have implemented AWS Lifecycle Policy in S3 for Following Purposes

Deleting the Multiple Versions of Same File
Deleting the Athena Output Files After 24 Hours

With the Help of this Lifecycle Policy we can save the unnecessary Cost by Deleting the unnecessary Files

That has Multiple Version Because Some times the Same Files Can have Version More then 10

Ex : A File that Contains 10 or more Copies of itself which is no longer Required but Finding and Delete manually is not possible because of the quantity

Version Ex images of single object for Reference:

Before:

After: Lifecycle Policy

Bucket Metrics After Implementing Lifecycle Policy

Note: Strict Lifecycle Policy Can result Low Spend on S3 Storage

Current Lifecycle Policy for object Version Control on prod.sprighub.ai and its backup prod.sprighub.ai-crr are Following

Note: We are Currently saving 7 non-Current Versions

We implemented the AWS Lifecycle Policy for S3 Bucket for Deletion of Athena Output File

Because AWS Athena Documentation is Recommends to do so unless or until you need to use the Same queries Result file in Future.

We implemented the Policy for output Files in of Athena to be Deleted After 24 Hours After Creations

We have Tested the Athena output Files to be Deleted From as Specific Folder inside the

dev.sprighub.ai and dev.sprighub.io named as

Test-Athena-By-Pawan-TBD/

Implemented the Following Lifecycle Policy

After End of Everyday The Previous Day Files has been Deleted

Ex:

Every time when we run a Athena Query Created Two Files But those Files are never Been Deleted own their own and takes the space in S3 Bucket but implementation of lifecycle policy will help to delete those files after one day

dev.sprighub.ai/Test-Athena-By-Pawan-TBD

Screenshot from 29-04-2022 to 02-05-2022