This document is for support use. It may not be complete or final.
Installation via the VAST Web UI enables you to choose from a few predefined scheme files or to upload a scheme file that you customized for the installation. After the scheme is loaded, the VAST Web UI enables you to change the values for the parameters that you included in the scheme file. It does not enable you to add any parameters that were not included in the scheme file.
This article is intended to guide you in selecting or creating a scheme file that you can use with the installer to set up the cluster.
-
Identify which of the scheme examples below is the closest to the type of installation that you are doing. Copy and paste the sample file content from that example into your preferred editor.
-
Customize the content with the help of the customization notes from your chosen example and the full format and parameters description in Scheme File Format and Parameters.
-
Save the file with a .yaml extension.
Identify the example below that closest matches the installation you're doing. Use the example content as a boilerplate.
This type of installation has:
-
One pair of 16 port Ethernet switches
-
Client data network, VAST internal network and management network all running on Ethernet
-
Management out of band
-
One NIC per CNode, with both interfaces used for both VAST internal and client data traffic.
Example Scheme Content
This example is identical to the content of the 2x16_switch_eth_only_mgmt_outbound scheme file. If this needs no customization for the installation, you can select this one in the installer without creating and uploading a custom scheme file.
switch_conf: num_of_switches: 2 num_of_ports: 16 roles: - spine spine: isl_ports: ipl_ports: [13,14] external_ports: [15,16] cnode_ports: [7/2,8/2,9/2,10/2,11/2,12/2] dnode_ports: [1,2,3,4,5,6] host_conf: mode: eth mgmt: outband cnode: ext_interface: eno1 # mgmt interface technician_interface: eno2 port_mode: eth eth_mtu: 9216 internal_interfaces: external_interfaces: internal_virtual_interfaces: dnode: ext_interface: ens6 # mgmt interface technician_interface: ens4 port_mode: eth eth_mtu: 9216 internal_interfaces: external_interfaces: internal_virtual_interfaces:
Customization Notes
-
Interface names vary depending on the model of CNode you are using. Therefore, you may need to change
ext_interface
andtechnician_interface
.Tip
You may need to SSH to one of the CNodes and DNodes and run
ip link
to verify or identify interface names. -
If the MTU on the client data network is greater than 9000, you'll need to set eth-mtu the CNodes and DNodes to the MTU of the client data network.
-
Notice that
internal_interfaces
,external_interfaces
andinternal_virtual_interfaces
are unspecified. These interface assignments are inferred automatically. When there is one NIC per CNode the inference is expected to be correct. -
VAST internal data is isolated from the client data network using a VLAN. This VLAN is 69, by default. In case of a conflicting use of the default VLAN (69), add the
data-vlan
parameter to the CNode and DNode configurations. Specify a different VLAN that is not already used on the client network.
This type of installation has:
-
Client data network running on Infiniband, VAST internal network running on Ethernet.
-
Two NICs per CNode one connected to the client data network and one connected to the VAST internal network.
Example Scheme Content
In this example, management is out of band and there is one pair of 16 port switches. Since there is only one switch pair, there is only one switch role (spine).
This example is similar to the content of the predefined scheme 2X16_switch_mixed_mgmt_outbound.yaml. We added ib-mode
to set a non-default mode for the Infiniband ports and ib-mtu
to set the MTU of the client data network. We also set port-mode
so that it configures one NIC in Ethernet mode for the VAST internal network and the other NIC in Infiniband mode for the client data network, as needed.
switch_conf: num_of_switches: 2 num_of_ports: 16 roles: - spine spine: isl_ports: ipl_ports: [13,14] external_ports: [15,16] cnode_ports: [7/2,8/2,9/2,10/2,11/2,12/2] dnode_ports: [1,2,3,4,5,6] host_conf: mode: mixed mgmt: outband cnode: ext_interface: eno1 # mgmt interface technician_interface: eno2 port_mode: internal=eth,external=ib eth_mtu: 9216 ib_mode: datagram ib_mtu: 2048 internal_interfaces: ens785f0,ens785f1 external_interfaces: ib0,ib1 internal_virtual_interfaces: ens785f2,ens785f3 dnode: ext_interface: ens6 # mgmt interface technician_interface: ens4 port_mode: eth eth_mtu: 9216
Customization Notes
-
The interface names in the example may not be correct for the CNode model that you're installing.
You will probably need to identify the names of interfaces on the CNodes and DNodes. You can do this by SSHing to one of the CNodes or DNodes and running the
ip link
command on the OS to identify interface names. -
The default IB mode is connected. Customers often require
datagram
, as specified in the example. -
When specifying port mode, note that you need to set port mode differently for each NIC. The order is according to PCI enumeration. In the example, we have
internal=eth,external=ib
. If the NICs are ordered the other way round, switch it toexternal=ib,internal=eth
-
DNode port mode should be set to the transport protocol of the VAST internal network (usually Ethernet for this type of installation), as in the example.
In a dual switch pair Ethernet installation, there are four 32 port switches, with one pair configured in the spine role for connectivity to the client data network, and the other pair connected only to the VAST internal network.
Both spine and leaf roles are declared in the scheme file and the port assignments for each role are specified as shown below.
Example Scheme Content
In this example, the networking is assumed Ethernet only and management is out of band.
switch_conf: roles: - spine - leaf num_of_switches: 4 num_of_ports: 32 spine: isl_ports: [23,24,25,26,27,28,29,30] ipl_ports: [31,32] external_ports: [15,16,17,18,19,20,21,22] cnode_ports: [9/2,10/2,11/2,12/2,13/2,14/2] dnode_ports: [1,2,3,4,5,6,7,8] leaf: isl_ports: [23,24,25,26,27,28,29,30] ipl_ports: [31,32] external_ports: cnode_ports: [9/2,10/2,11/2,12/2,13/2,14/2,15/2,16/2,17/2,18/2] dnode_ports: [1,2,3,4,5,6,7,8] host_conf: mode: eth mgmt: outband cnode: ext_interface: eno1 # mgmt interface technician_interface: eno2 port_mode: eth eth_mtu: 9216 internal_interfaces: external_interfaces: dnode: ext_interface: ens6 # mgmt interface technician_interface: ens4 port_mode: eth eth_mtu: 9216 internal_interfaces:
This type of installation has:
-
Unmanaged IB switches, negating the need for switch configuration
-
Client data network and VAST internal network both running on Infiniband,
-
Management inband
-
One NIC per CNode
Example Scheme Content
host_conf: mode: ib mgmt: inband cnode: ext_interface: eno1 # mgmt interface technician_interface: eno2 port_mode: ib ib_mode: datagram ib_mtu: 2044 internal_interfaces: external_interfaces: dnode: ext_interface: ens6 # mgmt interface technician_interface: ens4 port_mode: ib ib_mode: datagram ib_mtu: 2044 internal_interfaces:
Customization Notes
-
By default, VAST uses the following subnets: 172.16.1.x, 172.16.2.x, 172.16.3.x. In an IB configuration, there can be an IP address collision with these subnets. In such cases, add the template parameter and specify an alternate template for the subnets.
-
The interface assignments will be inferred automatically.
The scheme file is a follows the following format:
<section>: <parameter>: <value> <parameter>: <value> ... <section>: <parameter>: <value> <parameter>: <value> ...
Two sections can be included: switch_conf for switch configurations (not needed if, for example, the cluster will use unmanaged Infiniband switches) and host_conf needed for every installation for host configuration.
Each section should contain any of the parameters detailed in the following tables that are relevant to the installation. (Additional configurations, such as IP addresses, gateways etc. are provided through the later screens in the installer and not through the scheme file.)
Switch Configuration Parameters
The installer runs the switch_conf.py script on all switches
Comments
0 comments
Please sign in to leave a comment.