Uploading logs to VAST support bucket

Sometimes you need to get data to VAST for troubleshooting purposes. This data may be in the form of ad-hoc files or traces we've requested or it may be in the form of a generated support bundle that you create using the VMS.

There are many toolsets which can be used for getting files to VAST. We'll discuss a few here:

• s3cmd (more common, python based)
• AWS CLI (this ships with our C-nodes by default)
• 'mc' (newer, faster, golang based)
• VMS itself for support dumps

MC is preferred over s3cmd if you are attempting to upload files which are larger than 1GB, as it effectively leverages multiPartUploads in parallel. S3cmd, while it supports multiPartUploads, does so in a serial fashion, which is less desirable. AWS CLI works just fine as well. And of course if you generated the support dump via the VMS, you can push directly to VAST from the VMS.

Authentication

In all cases you'll need an AWS S3 access key in order to write to the VAST support bucket. Don't worry, the key you will use allows for write only access. No one outside of VAST can see what you've uploaded to the VAST support bucket - s3://vast-support.

The AWS S3 access key and secret is:

aws_access_key_id = AKIARTN2ZMSY4TBYMPNM
aws_secret_access_key = xzmAX+lFZSqH1/w03WuANqgj1YUhxWZ+iaXMMTAM

Proxies

If you do not have direct access to the internet from the host you are running your command line tools from, you may need to specify proxy settings. On Linux systems, that would be the following:

export http_proxy="http://PROXY_SERVER:PORT"
export https_proxy="https://PROXY_SERVER:PORT"

Using the VMS

If you've just created a support bundle, the most convenient way to upload the bundle is to look at the list of the bundles in the  VMS in the support panel and then click the little up arrow icon. That will ask you for your AWS key and secret as well as the target S3 bucket. Go ahead and specify the key and secret above and vast-support for the bucket. For example:

After specifying the information, click upload and the VMS will start working on the upload. It may take a while.  You can monitor this by looking at the activities page. The file will be placed in s3://vast-support/<CLUSTER NAME>/. The cluster name is the VAST cluster name displayed by the VMS in the  upper left.

Note: the VMS bundle upload feature gets information from the VAST callhome configuration panel - including proxy information - in order to execute the upload.

Using s3cmd

Install s3cmd

yum install -y s3cmd

Use the access key shown above and put it into a file such as vast-upload.s3cfg, for example:

[default]
access_key =AKIARTN2ZMSY4TBYMPNM
secret_key = xzmAX+lFZSqH1/w03WuANqgj1YUhxWZ+iaXMMTAM

Upload like so, substituting your organization for <company-name>

s3cmd -c vast-upload.s3cfg put /path/to/file/upload.file.tgz s3://vast-support/<company-name>/upload.file.tgz

Using minio client (mc) for large files

Install On OSX

brew install minio/stable/mc

Install On Linux

wget https://dl.minio.io/client/mc/release/linux-amd64/mc
chmod +x mc

Register S3 target and specify the access key and secret key mentioned earlier.

mc config host add vast https://s3.amazonaws.com $access_key $secret_key

In order to upload a file

mc cp /path/to/big.file.tar vast/vast-support/<company-name>/upload.file.tgz

Use AWS CLI

Note: the AWS CLI is installed by default with the VAST C-nodes in /usr/bin/aws.

To use the AWS CLI, install it on your platform and then set the environment variables for the S3 credentials. 

export AWS_ACCESS_KEY_ID=AKIARTN2ZMSY4TBYMPNM
export AWS_SECRET_ACCESS_KEY=xzmAX+lFZSqH1/w03WuANqgj1YUhxWZ+iaXMMTAM

Then run the AWS CLI using that credential file to upload the file, following the same conventions as discussed earlier. 

aws s3 cp /path/to/file/upload.file.tgz s3://vast-support/<company-name>/upload.file.tgz