S3 Client Configuration and Features

For an existing view with an S3 bucket:

When creating a new view:

For an existing view with an S3 bucket:

For example, this command enables versioning on a bucket whose view has ID 32:

$ view modify --id 32 --s3-versioning

When creating a new view:

Invoke the PutBucketVersioning S3 command to set the versioning state to Enabled.

When you upload an object to a version-enabled bucket, a unique version ID is automatically added to the object .

Each version of the object is stored as '<object-name>/<version-number>' where <object-name> is the object key and <version-number> is the version number. When listing objects, the S3 service returns objects in ascending order. Newer versions of objects have lower numbers than older versions, so the latest version is listed first.

To list all versions of a given object, send a GET Bucket request to list the object's version directory. Specify the directory by setting the prefix parameter to the object key.

A simple GET Object request retrieves the current version of an object.

To retrieve a specific version, you need to specify its version ID. Set the versionId to the ID of the version of the object you want to retrieve and send a GET Object versionId request.

A simple HEAD Object request retrieves metadata of the current version of an object.

To retrieve the metadata of a specific version of an object, send a HEAD Object versionId request, specifying the version ID of the object you want to retrieve.

A DeleteObject request for a versioned object does not delete it. It leaves previous versions of the object in the object version directory and creates a delete marker. A delete marker is an object that indicates that the object has been deleted. It has a key name (or key) and version ID like any other object but it is otherwise not a regular object: It does not have data associated with it nor does it have an access control list (ACL). It is not retrieved in a GET Object request or returned in a GET Bucket request.

You can delete the delete marker by sending a DeleteObject versionId request with the delete marker's version ID. This has the effect of undoing the object deletion.

To delete multiple versions of a versioned object, send a DeleteObjects request with an array of object identifiers (key, versionid)

The following operations restore a previous version of an object:

Suppose we upload the object myfile to the bucket mybucket using the S3cmd client:

[centos@host~]$ s3cmd put myfile s3://mybucket/
upload: 'myfile' > 's3://mybucket/myfile' [1 of 1]
10485760 of 10485760 100% in 0s 113.09 MB/s done

Normally, a request to delete the bucket would fail because the bucket is not empty. However, making a DeleteBucket request and specifying the x-amz-delete-contents header for the trash folder, the bucket and its content are deleted.

The following client session of the Boto3 AWS SDK for Python connects to the cluster's S3 service over HTTP at one of the cluster's CNode virtual IPs (198.51.100.255), authenticates with a VMS generated access key pair and deletes the bucket via the trash folder:

def set_delete_contents_header(request, **kwargs):
    request.headers.add_header('x-amz-delete-contents', 'true')

sess = boto3.session.Session()
s3 = sess.client(service_name='s3',
                 region_name='vast-1',
                 use_ssl=False, 
                 endpoint_url='http://198.51.100.255'
                 aws_access_key_id='YAUY93A7Q91SO2M07BWZ',
                 aws_secret_access_key='NWzyNIqIscgVZlaHqBzBWaejMDHSHbZuXDxbE2Yp'
                 config=boto3.session.Config(
                      signature_version='s3v4'
                      s3={'addressing_style': 'path'}
                  )
                )
s3.meta.events.register('before-sign.*.*', set_delete_contents_header)
s3.delete_bucket(Bucket='mybucket')

Following the deletion, no buckets are found:

[centos@host ~]$ s3cmd ls
[centos@host ~]$

Suppose two objects a/b/c/object1 and a/b/d/object2 are created in an existing bucket mybucket using the s3cmd client:

[centos@host ~]$ s3cmd put 1KB_random_file s3://mybucket/a/b/c/object1
upload: '1KB_random_file' -> 's3://mybucket/a/b/c/object1'  [1 of 1]
 1024 of 1024   100% in    0s    15.16 KB/s  done
[centos@host ~]$ s3cmd put 1KB_random_file s3://mybucket/a/b/d/object2
upload: '1KB_random_file' -> 's3://mybucketheaders/a/b/d/object2'  [1 of 1]
 1024 of 1024   100% in    0s    17.13 KB/s  done

These objects can be moved to trash by deleting the common object key prefix a/b, making a DeleteObject request specifying the x-amz-delete-contents header.

Using a client instance of the Boto3 AWS SDK for Python, contacting the S3 service over HTTP at one of the CNode virtual IPs (198.51.100.255), authenticating with a VMS generated access key pair, the request might look like this:

def set_delete_contents_header(request, **kwargs):
    request.headers.add_header('x-amz-delete-contents', 'true')

sess = boto3.session.Session()
s3 = sess.client(service_name='s3',
                 region_name='vast-1',
                 use_ssl=False, 
                 endpoint_url='http://198.51.100.255'
                 aws_access_key_id='YAUY93A7Q91SO2M07BWZ',
                 aws_secret_access_key='NWzyNIqIscgVZlaHqBzBWaejMDHSHbZuXDxbE2Yp'
                 config=boto3.session.Config(
                      signature_version='s3v4'
                      s3={'addressing_style': 'path'}
                  )
                )
s3.meta.events.register('before-sign.*.*', set_delete_contents_header)
s3.delete_object(Bucket='mybucket', Key='a/b')

Following the deletion, the objects can no longer be listed:

[centos@host~]$ s3cmd ls s3://mybucket/a/b
[centos@host ~]$