The basic units of S3 storage are objects. Every object is stored in a bucket. Buckets organize the S3 namespace at the highest level.
Objects consist of object data and metadata. The metadata is a set of name-value pairs that describe the object. These include some default metadata, such as the date last modified, and standard HTTP metadata, such as Content-Type. You can also specify custom metadata at the time the object is stored. An object is uniquely identified within a bucket by a key.
Although the S3 structure is flat, consisting of buckets with no hierarchy of sub-buckets and sub-folders, the '/' character acts as a delimiter from which a hierarchy can be inferred from object keys. Object listings can be filtered by any prefix that appears before the delimiter. For example, an object with key "2019-07-02/AboutVASTS3.rtf" would be returned by an object listing filtered by the prefix "2019-07-02/".
Objects can be up to one TB in size and up to two KB in metadata.
The following rules apply to object names:
Object names must be up to 1024 characters long.
Object names can contain up to 200 '/' characters.
The following rules apply to bucket names:
Bucket names must be unique across all existing bucket names on the cluster.
Bucket names must comply with DNS naming conventions.
Bucket names must be at least 3 and no more than 63 characters long.
Bucket names must not contain uppercase characters or underscores.
Bucket names must start with a lowercase letter or number.
Bucket names must be a series of one or more labels. Adjacent labels are separated by a single period (.).
Bucket names can contain lowercase letters, numbers, and hyphens. Each label must start and end with a lowercase letter or a number.
Amazon S3 is a REST service. VAST supports a subset of Amazon S3's REST APIs. Using REST, you can use HTTP requests to perform operations on the S3 service, such as create, fetch, and delete buckets and objects.
The endpoint for the S3 service can be any of the cluster's Virtual IPs. VAST Cluster can listen to S3 service requests on HTTP or on HTTPS.
Users require an access key and secret key in order to make authenticated requests. Users can create buckets and delete buckets only if they are assigned special permissions for these actions. You can grant users keys and specific permissions to create and delete buckets via the VAST Web UI or via the VAST CLI.
Every interaction with the S3 service is either authenticated or anonymous. Authentication is a process of verifying the identity of the requester trying to access the service. Authenticated requests must include a signature value that authenticates the request sender. VAST Cluster supports AWS signature version 2 and AWS signature version 4.
If you are using Boto 3 or S3cmd, the libraries compute the signature from the keys you provide. However, if you make direct REST API calls in your application, you must write the code to compute the signature and add it to the request.
You can manage access to buckets and objects using Access Control Lists (ACLs). See S3 Access Control Lists (ACLs) for more information.