Configuring Big Catalog

By default, Big Catalog is disabled. Enabling the big catalog triggers a background indexing task. When Big Catalog is enabled, snapshots of the Element Store are taken periodically, from which the Big Catalog is indexed asynchronously.

You can modify the periodic snapshot schedule. By default, a snapshot is taken every 30 minutes and a copy of that snapshot is retained for 30 minutes. The Big Catalog table is also retained for 30 minutes.

You can also add user defined attributes to be indexed.

Configuring Big Catalog from the VAST Web UI

From the left navigation menu, select Settings and then Big Catalog.
Slide the Enable slider to the right to enable Big Catalog (or to the left to disable Big Catalog).

Modify the default periodic snapshot schedule, if desired:

Note

Changes to retention periods apply to snapshots created after the change.

Save new catalog copies every	Sets how frequently to snapshot the file system and create a new catalog snapshot. Enter the number of time units in the Value field and select a time unit from the Period dropdown. Note The minimum interval is 15 minutes.
Keep catalog copies for	Sets how long to save each copy of the Big Catalog table. Enter the number of time units in the Value field and select a time unit from the Period dropdown.
Starting from	The date and time from which you want to start.
Store filesystem snapshots	Keep this setting enabled if you want to retain the snapshots from which Big Catalog is indexed.
Keep filesystem snapshots for	Sets how long to retain filesystem snapshots . Enter the number of time units in the Value field and select a time unit from the Period dropdown.

In the User defined attributes area, you can explicitly index S3 tags and metadata attributes that you use to categorize buckets and objects. When you can define a tag as a user defined attribute, a dedicated column is created named tag_<key>. The values of tags with the specified key name are stored in the dedicated column.

Note

All S3 tags are stored as key-pair values in the user_tags column, regardless of whether user defined attributes are specified.

To define an attribute:
1. From the Type dropdown, select a type of user defined attribute:
  - Tag. Select this value to define S3 tag keys as attributes to be indexed. After the next snapshot, a column namedtag_<key> will store the values of all S3 tags with the specified key.
  - Metadata. Select this value to define user-defined metadata keys as attributes to be indexed. After the next snapshot, a column named metadata_<key> will store the values of all S3 metadata attributes with the specified key. S3 metadata key value pairs can be assigned to objects when they are uploaded using a PutObject request. The metadata names must begin with x-amz-meta- to distinguish them from other HTTP headers.
2. In the Column Name field, enter the tag key that you want to create a column for. The value of each tag that has this key will be indexed in the created column.
3. Click Add. The attribute is added. This attribute will be indexed.
Click Save to save your changes.

A background indexing task is now triggered and a restore point named Big catalog stream (point #) on the protected path Big catalog stream is created. When the progress of the restore point reaches 100%, Big Catalog is indexed and ready for querying. You can watch the restore point progress from the Restore Points tab of the Data Protection page.