Enabling S3 User Access
Any client sending remote procedure calls (RPCs) via S3 requires an S3 access key pair in order to be authenticated. S3 access key pairs can be generated for any user via the VAST Web UI or VAST CLI.
Creating and deleting buckets require special permissions, which can also be assigned to each user via the VAST Web UI or VAST CLI.
To manage S3 access key pairs and permissions, see Managing User S3 Permissions.
The basic units of S3 storage are objects. Every object is stored in a bucket. Buckets organize the S3 namespace at the highest level.
Objects consist of object data and metadata. The metadata is a set of name-value pairs that describe the object. These include some default metadata, such as the date last modified, and standard HTTP metadata, such as Content-Type. You can also specify custom metadata at the time the object is stored. An object is uniquely identified within a bucket by a key.
Although the S3 structure is flat, consisting of buckets with no hierarchy of sub-buckets and sub-folders, the '/' character acts as a delimiter from which a hierarchy can be inferred from object keys. Object listings can be filtered by any prefix that appears before the delimiter. For example, an object with key "2019-07-02/AboutVASTS3.rtf" would be returned by an object listing filtered by the prefix "2019-07-02/".
Objects can be up to one TB in size and up to two KB in metadata.
Rules for Object Naming
The following rules apply to object names:
Object names must be up to 1024 characters long.
Object names can contain up to 200 '/' characters.
Object names must not have a leading slash. For example, the following is not supported:
Object names must not have a trailing slash. For example, the following is not supported:
Object names must not have a double slash. For example, the following is not supported:
Rules for Bucket Naming
The following rules apply to bucket names:
Bucket names must be unique across all existing bucket names on the cluster.
Bucket names must comply with DNS naming conventions.
Bucket names must be at least 3 and no more than 63 characters long.
Bucket names must not contain uppercase characters or underscores.
Bucket names must start with a lowercase letter or number.
Bucket names must be a series of one or more labels. Adjacent labels are separated by a single period (.).
Bucket names can contain lowercase letters, numbers, and hyphens. Each label must start and end with a lowercase letter or a number.
Amazon S3 is a REST service. VAST supports a subset of Amazon S3's REST APIs. Using REST, you can use HTTP requests to perform operations on the S3 service, such as create, fetch, and delete buckets and objects.
You can use any toolkit that supports HTTP to use the REST API. Read our VAST Data-specific guidelines for using the S3cmd command line tool or the Boto 3 SDK for Python.
S3 Service Endpoint
The endpoint for the S3 service can be any of the cluster's Virtual IPs. VAST Cluster can listen to S3 service requests on HTTP or on HTTPS.
Security Credentials and Permissions
Users require an access key and secret key in order to make authenticated requests. Users can create buckets and delete buckets only if they are assigned special permissions for these actions. You can grant users keys and specific permissions to create and delete buckets via the VAST Web UI or via the VAST CLI.
Every interaction with the S3 service is either authenticated or anonymous. Authentication is a process of verifying the identity of the requester trying to access the service. Authenticated requests must include a signature value that authenticates the request sender. VAST Cluster supports AWS signature version 2 and AWS signature version 4.
If you are using Boto 3 or S3cmd, the libraries compute the signature from the keys you provide. However, if you make direct REST API calls in your application, you must write the code to compute the signature and add it to the request.
Managing Access to Data
You can manage access to buckets and objects using Access Control Lists (ACLs). See S3 Access Control Lists (ACLs) for more information.
It is possible to grant 'S3 superuser' permission to a given user. This gives the user full access to buckets and objects, overriding ACLs. For information about how to do this, see Managing User S3 Permissions.
You can set a versioning state on a bucket and get information on the state of the bucket's versioning. You can also retrieve metadata about the versions of all the objects in a bucket. See Supported S3 API Operations for more information.
Article is closed for comments.