Articles in this section

Use AWS CLI and S3cmd with Vast

Andy Pernsteiner
  • Updated
Follow

Note: for more complete information, go here :First Steps to Working with S3

 

This article will be deprecated.

 

 

Here we provide a few simple examples of using the AWS CLI and the S3cmd line tools with VAST. The tools are very similar in behavior so we describe both together.

If you need assistance using these tools, or would like to use a different tools, please let us know and we'll be happy to help.

 

Create Credentials

To obtain the needed S3 credentials, use VMS to create them for the target user. The steps can be found in the official documentation.

 

Create Configuration File

S3cmd

s3cmd needs a configuration file that specifies the credentials as well as the endpoint information. To make a config file for s3cmd, name it something like vast.s3cfg. Put the following in, replacing IP_ADDRESS with one of the VAST VIPs (don't specify a port!).  And of course specify the access key and secret key you created earlier.

 

[default] 
access_key = REPLACE
secret_key = REPLACE
host_base = IP_ADDRESS
host_bucket = IP_ADDRESS
use_https = False
server_side_encryption = False
signature_v2 = False
signurl_use_https = False
encrypt = False

 

AWS CLI

The AWS CLI also works with VAST. First of course you'll need a credentials file. Normally this is in ~/.aws/credentials, but you can override the location by specifying the environment variable AWS_CONFIG_FILE. If you already have a configuration file, you can add a new stanza just for VAST. If you do not have a configuration file, let the AWS CLI create it for you by running this command and answering the questions. You can use the defaults for everything if you wish except of course provide the VAST access key and secret key when asked.

aws configure

If you already have a credentials file, you can easily add a new stanza for VAST. The resulting file will look something like this:

cat ~/.aws/credentials 
[default]
aws_access_key_id = xxxxxxx
aws_secret_access_key =xxxxxx
[vast]
aws_access_key_id = VASTACCESSKEY
aws_secret_access_key = VASTSECRETKEY

 If you've chosen to create a VAST specific profile for the AWS CLI then you'll need to specify --profile vast when using the command. Our examples will assume that.

You'll notice that unlike the s3cmd case we have not specified the IP address for VAST above, nor anything about SSL certificate verification. We will instead specify that on the command line when running the AWS CLI.

Sample Commands

Now that we've setup our environment, let's go ahead and try some simple operations.

List the root directory

s3cmd -c vast.s3cfg ls s3://

or 

aws --profile vast --no-verify-ssl --endpoint-url http://IP_ADDRESS s3 ls s3://

 

Make a bucket to work in

s3cmd -c vast.s3cfg mb s3://mybucket

or

aws --profile vast --no-verify-ssl --endpoint-url http://IP_ADDRESS s3 mb s3://mybucket

 

Put a file

s3cmd -c vast.s3cfg put vast.s3cfg s3://mybucket/

or

aws --profile vast --no-verify-ssl --endpoint-url http://IP_ADDRESS s3 cp vast.s3cfg s3://mybucket

 

List the new bucket

s3cmd -c vast.s3cfg ls s3://mybucket/

or

aws --profile vast --no-verify-ssl --endpoint-url http://IP_ADDRESS s3 ls s3://mybucket/

 

 

 

Related articles

Comments

0 comments

Please sign in to leave a comment.