As with any storage system, capacity in an important concept to understand. In VAST there are three types of capacity to consider: logical, physical, and usable. Each will be explained in depth, but briefly:

• logical capacity - how much data can be written to VAST. This is VAST's prediction of how much can be written to the system. This amount of data is what applications actually write. Basically if an application thinks it wrote 1GB, that is this number. VAST adjusts the logical capacity by considering various overheads and data reduction.
• usable capacity - how much data can be written to VAST assuming no data reduction. This number is how much data can be written to VAST after considering VAST overheads (such as data protection) but not considering data reduction. Another way of thinking about this number is this: if data reduction is none, then logical capacity equals usable capacity. If there is some data reduction, logical capacity is larger than usable capacity.
• physical capacity - the literal physical storage space reported by the drives after an initial 8% space reservation on system install. The amount of data that can be written to VAST if there is no data reduction (aka usable capacity) is always less than this amount because of the overhead introduced by VAST.

Until version 3.6, VAST reports physical capacity and logical capacity but not usable capacity. The product documentation explains the distinction between logical and physical capacity in the Dashboard, but that may leave some questions unanswered. In this article we go into a bit more depth on physical vs. usable capacity and how that relates to logical capacity.

Usable Capacity vs. Physical Capacity

When considering usable capacity, one starts with the raw physical capacity of the underlying physical storage and then subtracts overhead:

• Data Protection - VAST uses large erasure coding/RAID stripes to protect data. Parity blocks introduce overhead. The data protection overhead varies by the size of the erasure coding stripe which in turn varies based upon the number of storage enclosures (D-boxes) in a cluster. This overhead is shown incrementally as new data is written - by increasing the physical space used by more than was written after data reduction. The overhead varies between 2% and 11% as VAST varies the number of data blocks in the stripe between 36 and 146 - the number of parity blocks is always 4. In this article we will write that as D+P where D is the number of data blocks (36-146) and P is the number of parity blocks (4) per stripe. P is the overhead.
• Reserve - While VAST is not a log based file system in the traditional sense, VAST needs to keep a certain amount of physical space free in order to enable its background activities. The reserve will drop with each version of VAST.  Eventually our objective is to reduce the overhead to just 8%.  In keeping with that objective, the reserve is shown in two parts - at initial cluster bringup by reducing physical capacity shown by 8% and over time by increasing the physical space used by each write by an additional percentage that depends on the release.  The second value is shown incrementally because the goal is to reduce it to zero eventually. The release specific additional percentages are:
  • 3.0.x: 3% for a total of 11%
  • 3.2.x and later: 1% for a total of 9%

The fixed initial space reservation can be easily shown as in this diagram:

This is what is shown as the physical space upon cluster initialization. Please keep in mind that VAST prior to VAST 3.6 does not report usable capacity. Starting with VAST 3.6 VAST will report usable, physical, and logical capacity. The tables below are to help you plan how much you can store on a VAST system.

In tabular form, here is what the above means in terms of usable capacity - assuming no data reduction:

Note: half populated storage enclosures have slightly different overhead values and clusters where D-box HA / rack level protections are enabled have higher overhead values. Consult your VAST representative for the specifics.

Logical Capacity

The actual amount of that can written depends heavily on Data Reduction. Data reduction reduces the amount of usable capacity consumed by data which results in an increase in the logical capacity available for the data.  VAST uses three techniques for data reduction: global compression, global deduplication, and very soon global similarity compression. Data reduction reduces the amount of data written to some amount below what is logically written.  Data reduction is reported in VAST and readily visible in the dashboard - the value reported does not consider the overheads mentioned earlier that will slightly increase what is written to disk. In most cases data reduction exceeds the overheads by a large margin.

Logical capacity is a prediction of how much data can be stored assuming similar future data reduction ratios and no change in the overheads. Logical capacity is a much better predictor over time of the amount of data that can be stored in VAST than physical or usable capacity. The only caveat is that one must first store enough representative data into the system before the logical capacity prediction is meaningful.

The following diagram shows the relationship between logical and physical. The most important take away is that the total logical space (aka logical capacity) is calculated dynamically based upon the actual logical used, physical used, and total physical space.  Essentially the ratio of logical used to total logical space must always be the same as physical used to total physical space. Logical free is adjusted accordingly: logical free = logical space - logical used.

Example

The following diagram summarizes the VAST behavior with respect to the initial space reservation of 8%, data reduction, erasure coding, and incremental space reservation (1% today) for a single 1GB write:

Consider an example that should make this more concrete. Suppose a cluster contains four 675TB storage enclosures. The total physical space in VMS per enclosure will be reported as 92% of 675TB = 621TB - as mentioned earlier, that is the initial space reservation for internal overhead. Obviously for four enclosures the space reported is 621*4 = 2484TB. Then when a write of 1GB occurs that we will assume can be reduced by 50% using data reduction, the following will happen assuming a cluster with at least four full sized storage enclosures (enabling 146+4 stripes):

• The application logically writes 1GB and standard file system APIs report that 1GB was written
• Data reduction will reduce the actual data to be written to 500MB
• Erasure coding overhead will add in 2.8% (4/146) more physical space to be used
• Reserve overhead will add in 1% (as of 3.2) more physical space to be used
• Total bytes written is then 500MB x (100% + 2.8% + 1%) = 519MB
• Physical space used will increase by 519MB and physical space free will decrease by 519MB
• Logical space used will increase by 1GB
• Logical space free will be adjusted to reflect the new predicted logical storage space remaining. 

Quotas

Quotas in VAST are against logical space used. Thus when an application logically writes 1GB of data, their quota usage increases by 1GB. This is done intentionally as usable or physical used is largely meaningless to end users. Consider that one user could write 1GB of data and get no data reduction while second user might write 1GB of different data and get a large amount of data reduction because some third user already wrote similar data. But at some future time that third user might delete the similar data - it would make no sense to change the second user's quota usage based upon what some other user did.

Two caveats regarding quotas:

• VAST internally tracks file sizes using 32K increments. This means that the smallest quota increment is 32K. Thus a 1K file written to VAST will consume at most 1K of space (even less with data reduction) but the quota used will be 32K.
• There is going to be a product enhancement to optionally enable more accurate quota accounting but this will introduce the side effect of quotas not properly considering sparse files - essentially the quota usage for a sparse file will consider its full logical size, not the bytes written.

Capacity License for Usable Capacity

VAST allows for the licensing of usable capacity. If your system is licensed under those terms, the description in this article of usable capacity applies. In VAST 3.6, this will be clearly reported. For earlier versions this can be computed manually.  Please see table above to compute the amount of usable capacity that your system is capable of.