By default, Big Catalog is disabled. Enabling the big catalog triggers a background indexing task. When Big Catalog is enabled, snapshots of the Element Store are taken periodically, from which the Big Catalog is indexed asynchronously.
You can modify the periodic snapshot schedule. By default, a snapshot is taken every 30 minutes and a copy of that snapshot is retained for 30 minutes. The Big Catalog table is also retained for 30 minutes.
You can also add user defined attributes to be indexed.
From the left navigation menu, select Settings and then Big Catalog.
Slide the Enable slider to the right to enable Big Catalog (or to the left to disable Big Catalog).
Modify the default periodic snapshot schedule, if desired:
Changes to retention periods apply to snapshots created after the change.
In the User defined attributes area, you can explicitly index S3 tags and metadata attributes that you use to categorize buckets and objects. When you can define a tag as a user defined attribute, a dedicated column is created named
tag_<key>. The values of tags with the specified key name are stored in the dedicated column.
All S3 tags are stored as key-pair values in the
user_tagscolumn, regardless of whether user defined attributes are specified.
To define an attribute:
From the Type dropdown, select a type of user defined attribute:
Tag. Select this value to define S3 tag keys as attributes to be indexed. After the next snapshot, a column named
tag_<key>will store the values of all S3 tags with the specified key.
Metadata. Select this value to define user-defined metadata keys as attributes to be indexed. After the next snapshot, a column named
metadata_<key>will store the values of all S3 metadata attributes with the specified key. S3 metadata key value pairs can be assigned to objects when they are uploaded using a PutObject request. The metadata names must begin with
x-amz-meta-to distinguish them from other HTTP headers.
In the Column Name field, enter the tag key that you want to create a column for. The value of each tag that has this key will be indexed in the created column.
Click Add. The attribute is added. This attribute will be indexed.
Click Save to save your changes.
A background indexing task is now triggered and a restore point named Big catalog stream (point #) on the protected path Big catalog stream is created. When the progress of the restore point reaches 100%, Big Catalog is indexed and ready for querying. You can watch the restore point progress from the Restore Points tab of the Data Protection page.
To view the Big Catalog configuration, use bigcatalogconfig list.
To enable or disable Big Catalog and to modify the periodic snapshot schedule, use bigcatalogconfig modify.
To add, list or remove columns for user defined attributes, use bigcatalogindexedcolumn add, bigcatalogindexedcolumn list and bigcatalogindexedcolumn remove.
To remove a Big Catalog configuration and schedule, use bigcatalogindexedcolumn remove.
To create a Big Catalog periodic snapshot schedule, use bigcatalogconfig create:
Article is closed for comments.