Skip to content
  • There are no suggestions because the search field is empty.

How to set up immutable backups with object lock and lifecycle rules for backup tools without version-aware deletes

This article explains how to setup an immutable backup with object lock and how to remove expired backup data with lifecycle rules when the backup tool does not support version-aware deletes.

Immutable backups protect backup data from accidental deletion and ransomware attacks. They rely on two features of the S3 bucket: Versioning and object lock. When both are enabled, a delete request without a version ID does not remove any data. It only adds a delete marker, and the previous versions are kept, which may lead to infinitely growing storage consumption and discrepancies in the storage numbers reported. 

Some backup tools fully support immutable backups. They set the retention on each object and delete expired object versions themselves. Other backup tools support only part of this or none of it. They rely on the storage to set the retention, to remove expired object versions, or both.

This article describes how versioning, object lock, default retention, and lifecycle rules are combined for backup tools that do not fully support immutable backups. The examples use a 30-day immutability period, the eu-west-2 region, and a bucket named my-backup-bucket. These values should be replaced with the actual values of the setup.

How it works

The setup combines object lock and a lifecycle rule. Object lock provides the protection. The lifecycle rule cleans up expired objects.

The lifecycle rule period must be longer than the immutability period.

This ensures that an old version is only deleted after its protection has ended.
Example: immutability period of 30 days → lifecycle rule of 31 days or more.

For more detailed definitions, see Versioning and Object Lock and Object lifecycle management.

Prerequisites

The following access, tools, and information are needed before starting the setup:

Step 1: Create a bucket with versioning and object lock

Despite on the backup tools being used, immutable backups require a bucket with enabled versioning and object lock settings. The following instructions guide you on how to create a bucket and set it up correctly:

  1. Log in to the Impossible Cloud Storage console
  2. Click Add Bucket, on the top-right corner of the Buckets page.
  3. Input a Bucket Name and choose a storage Region, in this example we use eu-west-2.
  4. Enable Versioning. Versioning is required for Object Lock.
  5. Enable Object Lock. For detailed instructions, see Enabling Object Lock.
  6. Proceed to Step 2 before creating the bucket

Step 2 (optional): Set a default retention period on the bucket

Default retention applies the immutability period to every new object version uploaded to the bucket. This feature is only required if the backup tool does not manage Object Lock retention.

Consider the following when deciding to enable a default retention in the bucket, please c:

  • If the backup tool manages object retention but does not do version-aware deletes (e.g. Acronis Cyber Protect Cloud), leave Retention disabled on the bucket and set immutability period on the backup configuration
  • If the backup tool does not manage Object Lock retention and does not do version-aware delete (e.g. Synology Hyper Backup), enable Retention on the bucket and set the desired immutability period (e.g. 30 days) and mode (see Retention modes).

Always refer to the third-party tools documentation.

Step 3: Confirm bucket creation

Once the bucket configuration (i.e. versioning, object lock, and retention) is set. Finalize the bucket creation by clicking "Add."

Step 4: Apply the lifecycle rule

The use of the lifecycle rule in this case is to permanently delete old versions and leftover delete markers once the immutability period has ended. Lifecycle rules are applied through the S3 API with the AWS CLI to the created bucket.

1. Configure the AWS CLI

Run the following command and enter the access key, secret key, and region (eu-west-2 in this example):

aws configure --profile impossiblecloud

2. Create the lifecycle rule file

Create a file named lifecycle.json with the following content:

{
  "Rules": [
    {
      "ID": "expire-noncurrent-versions-after-immutability",
      "Status": "Enabled",
      "Filter": {},
      "NoncurrentVersionExpiration": {
        "NoncurrentDays": 31
      }
    },
    {
      "ID": "remove-expired-delete-markers",
      "Status": "Enabled",
      "Filter": {},
      "Expiration": {
        "ExpiredObjectDeleteMarker": true
      }
    },
    {
      "ID": "abort-incomplete-multipart-uploads",
      "Status": "Enabled",
      "Filter": {},
      "AbortIncompleteMultipartUpload": {
        "DaysAfterInitiation": 7
      }
    }
  ]
}

The file contains three rules that apply to all objects in the bucket:

  • Expire-noncurrent-versions-after-immutability deletes old (non-current) versions 31 days after they were deleted or replaced by the backup software.
  • Remove-expired-delete-markers removes delete markers that have no versions left behind them.
  • Abort-incomplete-multipart-uploads removes parts of unfinished uploads, for example from interrupted backup jobs.

Important: An Expiration rule with Days must not be added to a backup bucket. It would hide current backup files from the backup software and break backup chains.

3. Apply the lifecycle rule to the bucket

aws s3api put-bucket-lifecycle-configuration --bucket my-backup-bucket --lifecycle-configuration file://lifecycle.json --endpoint-url https://eu-west-2.storage.impossibleapi.net --profile impossiblecloud

A successful command returns no output. If any part of the file is invalid, nothing is stored. For error codes and supported options, see Object lifecycle management.

Result

The backups in the bucket are now immutable for 30 days, and old versions are removed automatically 31 days after they were deleted or overwritten by the backup software. Until the lifecycle rule removes them, these versions remain in the bucket and remain protected from any deletes or modifications.

For a backup tool that manages object retention but does not do version-aware deletes, the backup tool will maintain object retentions and the bucket lifecycle rules will clean after expired objects. For a backup tool that does not manage Object Lock retention and does not do version-aware delete, the default bucket retention as well as the bucket lifecycle rules maintain object retention as well as clean up expired object versions.