commit 5beef1386bec1159f15c4bd0b1fd6743f9ca6fa1 Author: tseed Date: Wed Oct 26 19:12:37 2022 +0100 initial commit diff --git a/RDIS_ESXi7_v2.md b/RDIS_ESXi7_v2.md new file mode 100755 index 0000000..d94b721 --- /dev/null +++ b/RDIS_ESXi7_v2.md @@ -0,0 +1,2750 @@ +RDIS ESXi7 v2 + +# Supporting Information + +## Jump host profiles + +> local ssh config file + +``` +#Initial account setup + Host robots + Hostname login.robots.university.ac.uk + #user ocf@robots.university.ac.uk + User ocf + #run custom script /usr/local/bin/changemypw + + #External jump host + Host rdis + HostName login.robots.university.ac.uk + ProxyJump nemesis + #full path for key when using sudo ssh -F + IdentityFile /home/tseed/.ssh/id_rsa + #IdentityFile ~/.ssh/id_rsa + Port 22 + User ocf + + #OCF setup jump host + Host rdis-20 + Hostname 129.67.94.40 + ProxyJump rdis + User ocfuser + IdentityFile ~/.ssh/id_rsa + Port 22 + #browser socks5 proxy @127.0.0.1:8080 + DynamicForward 8080 + + #OCF setup jump host profile for VCSA install + Host rdis-vsphere + Hostname 129.67.94.40 + ProxyJump rdis + User ocfuser + #full path for key when using sudo ssh -F + IdentityFile /home/tseed/.ssh/id_rsa + #IdentityFile ~/.ssh/id_rsa + Port 22 + #port forward to ESXi host API endpoint + LocalForward 443 10.0.1.21:443 +``` + +This config files was used with WSL, enabling the local forward port to be presented on the loopback of the windows laptop for the VCSA installer tool target, an equivalent putty config could be used. + +## Server ilom/ipmi + +> user: USERID +> pass: Password0 + +| Host | TSM Console | +| --- | --- | +| rdis-vmserver01 | https://10.0.1.11/#login | +| rdis-vmserver02 | https://10.0.1.12/#login | +| rdis-vmserver03 | https://10.0.1.13/#login | +| rdis-vmserver04 | https://10.0.1.41/#login | + +## ESXi Web console + +> user: root +> pass: Password0 + +| Host | IP | vSphere Console | +| --- | --- | --- | +| rdis-vmserver01 | 10.0.1.21/24 | https://10.0.1.21/ui/#/login | +| rdis-vmserver02 | 10.0.1.22/24 | https://10.0.1.22/ui/#/login | +| rdis-vmserver03 | 10.0.1.23/24 | https://10.0.1.23/ui/#/login | +| rdis-vmserver04 | 10.0.1.51/24 | https://10.0.1.51/ui/#/login | + +## Network interfaces + +To get to the ESXi shell from the lenovo TSM lom console: + +> https://kb.vmware.com/s/article/2004746 + +Use the following command to get the associations between mac address, physical interface and vmnic interface: + +> esxcli network nic list + +| | | | | | | | | +| --- | --- | --- | --- | --- | --- | --- | --- | +| **Server** | **NIC** | **MAC** | **vSwitch** | **Portgroup**
**VMkernel** | **Physical Switch Port** | **VLAN** | **Class** | +| rdis-vmserver01 | ilom | 3ce1.a1c8.108b | N/A | N/A IP = 10.0.1.11/24 | (Primary)
Ethernet1/5 | access vlan 2001 | out of band management | +| rdis-vmserver01 | vmnic0 | 0c:42:a1:d2:fb:72
0c42.a1d2.fb72 | Storage | DVP\_iSCSI\_A (VLAN 0 = native VLAN 2011)
vmk2 10.0.3.11/27
DVP\_iSCSI\_B (VLAN 0 = native VLAN 2011)
vmk3 10.0.3.43/27
DVP\_iSCSI\_C (VLAN 0 = native VLAN 2011)
vmk4 10.0.3.75/27
DVP\_iSCSI\_D (VLAN 0 = native VLAN 2011)
vmk4 10.0.3.107/27 | (Primary)
Ethernet1/49/1 | access vlan 2011 | iSCSI | +| rdis-vmserver01 | vmnic1 | 0c:42:a1:d2:fb:73
0c42.a1d2.fb73 | Guest | Guest (VLAN 0 = native 2012)
Additional port groups on *any VLAN for guest networks to be configured | (Primary)
Ethernet1/49/3 | trunk native vlan 2012 | VM-Guest | +| rdis-vmserver01 | vmnic2 | 0c:42:a1:d2:fa:ea
0c42.a1d2.faea | Storage | DVP\_iSCSI\_A (VLAN 0 = native VLAN 2011)
vmk2 10.0.3.11/27
DVP\_iSCSI\_B (VLAN 0 = native VLAN 2011)
vmk3 10.0.3.43/27
DVP\_iSCSI\_C (VLAN 0 = native VLAN 2011)
vmk4 10.0.3.75/27
DVP\_iSCSI\_D (VLAN 0 = native VLAN 2011)
vmk4 10.0.3.107/27 | (Primary)
Ethernet1/49/2 | access vlan 2011 | iSCSI | +| rdis-vmserver01 | vmnic3 | 0c:42:a1:d2:fa:eb
0c42.a1d2.faeb | Guest | Guest (VLAN 0 = native 2012)
Additional port groups on *any VLAN for guest networks to be configured | (Primary)
Ethernet1/49/4 | trunk native vlan 2012 | VM-Guest | +| rdis-vmserver01 | vmnic4 | 34:80:0d:2d:85:c8
3480.0d2d.85c8 | vSwitch0 / DVS_Management | Management Network (VLAN 0 = native VLAN 2001)
vmk0 10.0.1.21/24
External Management Network (VLAN 20)
vmk1 129.67.94.20/23 | (Primary)
Ethernet1/16 | trunk native vlan 2001 | ESX Mgmnt / HA/DRS | +| rdis-vmserver01 | vmnic5 | 34:80:0d:2d:85:d0
3480.0d2d.85d0 | vSwitch0 / DVS_Management | Management Network (VLAN 0 = native VLAN 2001)
vmk0 10.0.1.21/24
External Management Network (VLAN 20)
vmk1 129.67.94.20/23 | (Primary)
Ethernet1/17 | trunk native vlan 2001 | ESX Mgmnt / HA/DRS | +| rdis-vmserver02 | ilom | 3ce1.a1c8.1121 | N/A | N/A IP = 10.0.1.12/24 | (Primary)
Ethernet1/6 | access vlan 2001 | out of band management | +| rdis-vmserver02 | vmnic0 | 0c:42:a1:d2:fb:6a
0c42.a1d2.fb6a | Storage | DVP\_iSCSI\_A (VLAN 0 = native VLAN 2011)
vmk2 10.0.3.13/27
DVP\_iSCSI\_B (VLAN 0 = native VLAN 2011)
vmk3 10.0.3.45/27
DVP\_iSCSI\_C (VLAN 0 = native VLAN 2011)
vmk4 10.0.3.77/27
DVP\_iSCSI\_D (VLAN 0 = native VLAN 2011)
vmk4 10.0.3.109/27 | (Primary)
Ethernet1/50/1 | access vlan 2011 | iSCSI | +| rdis-vmserver02 | vmnic1 | 0c:42:a1:d2:fb:6b
0c42.a1d2.fb6b | Guest | Guest (VLAN 0 = native 2012)
Additional port groups on *any VLAN for guest networks to be configured | (Primary)
Ethernet1/50/3 | trunk native vlan 2012 | VM-Guest | +| rdis-vmserver02 | vmnic2 | 0c:42:a1:d2:fb:5e
0c42.a1d2.fb5e | Storage | DVP\_iSCSI\_A (VLAN 0 = native VLAN 2011)
vmk2 10.0.3.13/27
DVP\_iSCSI\_B (VLAN 0 = native VLAN 2011)
vmk3 10.0.3.45/27
DVP\_iSCSI\_C (VLAN 0 = native VLAN 2011)
vmk4 10.0.3.77/27
DVP\_iSCSI\_D (VLAN 0 = native VLAN 2011)
vmk4 10.0.3.109/27 | (Primary)
Ethernet1/50/2 | access vlan 2011 | iSCSI | +| rdis-vmserver02 | vmnic3 | 0c:42:a1:d2:fb:5f
0c42.a1d2.fb5f | Guest | Guest (VLAN 0 = native 2012)
Additional port groups on *any VLAN for guest networks to be configured | (Primary)
Ethernet1/50/4 | trunk native vlan 2012 | VM-Guest | +| rdis-vmserver02 | vmnic4 | 34:80:0d:2d:8e:80
3480.0d2d.8e80 | vSwitch0 / DVS_Management | Management Network (VLAN 0 = native VLAN 2001)
vmk0 10.0.1.22/24
External Management Network (VLAN 20)
vmk1 129.67.94.21/23 | (Primary)
Ethernet1/18 | trunk native vlan 2001 | ESX Mgmnt / HA/DRS | +| rdis-vmserver02 | vmnic5 | 34:80:0d:2d:8e:88
3480.0d2d.8e88 | vSwitch0 / DVS_Management | Management Network (VLAN 0 = native VLAN 2001)
vmk0 10.0.1.22/24
External Management Network (VLAN 20)
vmk1 129.67.94.21/23 | (Primary)
Ethernet1/19 | trunk native vlan 2001 | ESX Mgmnt / HA/DRS | +| rdis-vmserver03 | ilom | 3ce1.a1c8.109b | N/A | N/A IP = 10.0.1.13/24 | (Primary)
Ethernet1/7 | access vlan 2001 | out of band management | +| rdis-vmserver03 | vmnic0 | 0c:42:a1:d2:fb:7e
0c42.a1d2.fb7e | Storage | DVP\_iSCSI\_A (VLAN 0 = native VLAN 2011)
vmk2 10.0.3.15/27
DVP\_iSCSI\_B (VLAN 0 = native VLAN 2011)
vmk3 10.0.3.47/27
DVP\_iSCSI\_C (VLAN 0 = native VLAN 2011)
vmk4 10.0.3.79/27
DVP\_iSCSI\_D (VLAN 0 = native VLAN 2011)
vmk4 10.0.3.111/27 | (Primary)
Ethernet1/51/1 | access vlan 2011 | iSCSI | +| rdis-vmserver03 | vmnic1 | 0c:42:a1:d2:fb:7f
0c42.a1d2.fb7f | Guest | Guest (VLAN 0 = native 2012)
Additional port groups on *any VLAN for guest networks to be configured | (Primary)
Ethernet1/51/3 | trunk native vlan 2012 | VM-Guest | +| rdis-vmserver03 | vmnic2 | 0c:42:a1:d2:fb:7a
0c42.a1d2.fb7a | Storage | DVP\_iSCSI\_A (VLAN 0 = native VLAN 2011)
vmk2 10.0.3.15/27
DVP\_iSCSI\_B (VLAN 0 = native VLAN 2011)
vmk3 10.0.3.47/27
DVP\_iSCSI\_C (VLAN 0 = native VLAN 2011)
vmk4 10.0.3.79/27
DVP\_iSCSI\_D (VLAN 0 = native VLAN 2011)
vmk4 10.0.3.111/27 | (Primary)
Ethernet1/51/2 | access vlan 2011 | iSCSI | +| rdis-vmserver03 | vmnic3 | 0c:42:a1:d2:fb:7b
0c42.a1d2.fb7b | Guest | Guest (VLAN 0 = native 2012)
Additional port groups on *any VLAN for guest networks to be configured | (Primary)
Ethernet1/51/4 | trunk native vlan 2012 | VM-Guest | +| rdis-vmserver03 | vmnic4 | 34:80:0d:2d:8b:50
3480.0d2d.8b50 | vSwitch0 / DVS_Management | Management Network (VLAN 0 = native VLAN 2001)
vmk0 10.0.1.23/24
External Management Network (VLAN 20)
vmk1 129.67.94.22/23 | (Primary)
Ethernet1/20 | trunk native vlan 2001 | ESX Mgmnt / HA/DRS | +| rdis-vmserver03 | vmnic5 | 34:80:0d:2d:8b:58
3480.0d2d.8b58 | vSwitch0 / DVS_Management | Management Network (VLAN 0 = native VLAN 2001)
vmk0 10.0.1.23/24
External Management Network (VLAN 20)
vmk1 129.67.94.22/23 | (Primary)
Ethernet1/21 | trunk native vlan 2001 | ESX Mgmnt / HA/DRS | +| rdis-vmserver04 (DR) | ilom | 3ce1 a1c8 165b | N/A | N/A IP = 10.0.1.41/24 | (DR)
Ethernet1/3 | access vlan 2001 | out of band management | +| rdis-vmserver04 (DR) | vmnic0 | 0c:42:a1:d2:fb:66 | Storage | DVP\_iSCSI\_A (VLAN 0 = native VLAN 2011)
vmk2 10.0.3.17/27
DVP\_iSCSI\_B (VLAN 0 = native VLAN 2011)
vmk3 10.0.3.49/27
DVP\_iSCSI\_C (VLAN 0 = native VLAN 2011)
vmk4 10.0.3.81/27
DVP\_iSCSI\_D (VLAN 0 = native VLAN 2011)
vmk4 10.0.3.113/27 | (DR)
Ethernet1/49/3 | access vlan 2011 | iSCSI | +| rdis-vmserver04 (DR) | vmnic1 | 0c:42:a1:d2:fb:67 | Guest | Guest (VLAN 0 = native 2012)
Additional port groups on *any VLAN for guest networks to be configured | (DR)
Ethernet1/49/1 | trunk native vlan 2012 | VM-Guest | +| rdis-vmserver04 (DR) | vmnic2 | 0c:42:a1:d2:fb:62 | Storage | DVP\_iSCSI\_A (VLAN 0 = native VLAN 2011)
vmk2 10.0.3.17/27
DVP\_iSCSI\_B (VLAN 0 = native VLAN 2011)
vmk3 10.0.3.49/27
DVP\_iSCSI\_C (VLAN 0 = native VLAN 2011)
vmk4 10.0.3.81/27
DVP\_iSCSI\_D (VLAN 0 = native VLAN 2011)
vmk4 10.0.3.113/27 | (DR)
Ethernet1/49/2 | access vlan 2011 | iSCSI | +| rdis-vmserver04 (DR) | vmnic3 | 0c:42:a1:d2:fb:63 | Guest | Guest (VLAN 0 = native 2012)
Additional port groups on *any VLAN for guest networks to be configured | (DR)
Ethernet1/49/4 | trunk native vlan 2012 | VM-Guest | +| rdis-vmserver04 (DR) | vmnic4 | 34:80:0d:2d:8b:c8 | vSwitch0 / DVS_Management | Management Network (VLAN 0 = native VLAN 2001)
vmk0 10.0.1.51/24
External Management Network (VLAN 20)
vmk1 129.67.94.29/23 | (DR)
Ethernet1/8 | trunk native vlan 2001 | ESX Mgmnt / HA/DRS | +| rdis-vmserver04 (DR) | vmnic5 | 34:80:0d:2d:8b:d0 | vSwitch0 / DVS_Management | Management Network (VLAN 0 = native VLAN 2001)
vmk0 10.0.1.51/24
External Management Network (VLAN 20)
vmk1 129.67.94.29/23 | (DR)
Ethernet1/9 | trunk native vlan 2001 | ESX Mgmnt / HA/DRS | + +# Vmware hypervisor install + +> Use the lenovo ESXi image to ensure all network card drivers are included +> +> [https://vmware.lenovo.com/content/2020\_08/Lenovo\_Custom_ISO/7.0/](https://vmware.lenovo.com/content/2020_08/Lenovo_Custom_ISO/6.7u3/) + +Mount the ISO from the ilom KVM viewer, run through the ESXi installer using the RAID1 SSD. + +After install unmount the install media and enter the bios. + +## Enable TPM module from the bios and set boot from the UEFI partition + +![18199247e9dfa4058d2f36d9ed110ac5.png](_resources/8c22b2b544734b7a8c2f204a944887a2.png) + +Check TPM is set to version 2.0, if the option to switch to 1.2 is present the host is already running with TPM version 2.0 and does not need to be updated. + +![aa6a3c0784bad01ad9d86dc7768e6dbf.png](_resources/7dd61b90021547a984850d481765c16e.png) + +Change the first boot option to UEFI OS on M.2 RAID virtual disk. + +![f651a7140d45764b132f11d1aed5928a.png](_resources/676365faba3c4659b28cd12d11d10d42.png) + +![78fff1e872b833a53eb3b608213c3544.png](_resources/e144fba8e1ab4a40a68d6c73e7094d51.png) + +## Ensure the OCP adapter interfaces are assigned to the management vmkernel interface + +Configure Management Network: + +![4fc92cc4cc1ddd39577c7c2e4e8791e6.png](_resources/fc8dbdc2707641c591e3dd8e89d48d4a.png) + +Assign Network Adapters to the the Management Network: + +## ![cc9eabb84afce5e6370be212c38ece80.png](_resources/c58e9161c2b14fdabc65ec2520df7e90.png) + +![de39f76b1ecd7785e83d9b56aea3172b.png](_resources/41c26ff2a41e4ea2a7331606a19b541c.png) + +Configure IPv4 address: + +![fc5ae6e8e4c4e7e781551c211a43d236.png](_resources/d185f915d58a4db08e91defe9665e79a.png) + +| ESXI host | Management IP | +| --- | --- | +| rdis-vmserver01 | 10.0.1.21/24 | +| rdis-vmserver02 | 10.0.1.22/24 | +| rdis-vmserver03 | 10.0.1.23/24 | +| rdis-vmserver04 | 10.0.1.51/24 | + +Ensure the gateway is set to 0.0.0.0, this internal network is isolated. + +## ![7817bf895077a701dd7a043481372d6f.png](_resources/acf77b15bfe440aab3a023b9cf37ad5d.png) + +## ![ec92e5ff95a838fd9801194df33b36db.png](_resources/b7171ddc06bf4a2a978f9541bc0f130e.png) + +This default configuration creates an active-standby pair for vmnic4 and vmnic5 respectively with the default NIC teaming policy Routed based on originating port ID. + +# Setup switch ports + +| Switch | SSH IP | +| --- | --- | +| Primary site switch | 129.67.94.5 | +| DR site switch | 129.67.94.7 | + +- Ensure any physical switch trunk ports have are configured with a native VLAN configuration. +- Ensure management trunks allows VLAN 20 (Ext management), has native access to VLAN 2001 (Int Management) and allows VLAN 2002 (vMotion). +- Ensure storage trunks have native VLAN access to VLAN 2011. +- Ensure guest trunks allow all VLANs to ensure customer can extend networks over the passed over the "University RDIS Trunk" uplink. +- LACP configuration is applied to VMware guest ports. + +## Primary site switch + +> ssh admin@129.67.94.5 +> password: Password0 + +``` +#find switch port by mac address, this only works where the interface is configured with an ip +show mac address-table +show mac address-table address 3480.0d2d.85c8 + +NE1072T>show mac address-table address 3480.0d2d.85c8 + VLAN MAC Address Type Ports + -----+----------------+---------+-----------------------+ + 2001 3480.0d2d.85c8 dynamic Ethernet1/17 + +#where the interface is not yet configured in vmware, change the interface link speed in vmware console from auto/manual/auto and watch the switch logs to correlate the vmic label to the switch port. + +#setup ports to trunk mode or LACP trunk mode and assign vlan + +## VM management/vSphere traffic + +enable +configure + +# configure ports for management +interface Ethernet1/16-21 +no channel-group +switchport mode trunk +switchport trunk native vlan 2001 +switchport trunk allowed vlan 2001,2002,20 + +## SR635 #1 VM iSCSI + +# configure ports for VM iSCSI +interface Ethernet1/49/1 +no channel-group +no description +switchport mode trunk +switchport trunk native vlan 2011 +switchport trunk allowed vlan 2011 +description "SR635 #1 S2P0 iSCSI" +exit + +interface Ethernet1/49/2 +no channel-group +no description +switchport mode trunk +switchport trunk native vlan 2011 +switchport trunk allowed vlan 2011 +description "SR635 #1 S3P0 iSCSI" +exit + +## SR635 #2 VM iSCSI + +# configure ports for VM iSCSI +interface Ethernet1/50/1 +no channel-group +no description +switchport mode trunk +switchport trunk native vlan 2011 +switchport trunk allowed vlan 2011 +description "SR635 #2 S2P0 iSCSI" +exit + +interface Ethernet1/50/2 +no channel-group +no description +switchport mode trunk +switchport trunk native vlan 2011 +switchport trunk allowed vlan 2011 +description "SR635 #2 S3P0 iSCSI" +exit + +## SR635 #3 VM iSCSI + +# configure ports for VM iSCSI +interface Ethernet1/51/1 +no channel-group +no description +switchport mode trunk +switchport trunk native vlan 2011 +switchport trunk allowed vlan 2011 +description "SR635 #3 S2P0 iSCSI" +exit + +interface Ethernet1/51/2 +no channel-group +no description +switchport mode trunk +switchport trunk native vlan 2011 +switchport trunk allowed vlan 2011 +description "SR635 #3 S3P0 iSCSI" +exit + +## SR635 #1 VM Guest + +# configure ports for VM Guest +interface Ethernet1/49/3 +no channel-group +no description +switchport mode trunk +switchport trunk native vlan 2012 +switchport trunk allowed vlan all +description "SR635 #1 S2P1 VM-Guest" +exit + +interface Ethernet1/49/4 +no channel-group +no description +switchport mode trunk +switchport trunk native vlan 2012 +switchport trunk allowed vlan all +description "SR635 #1 S3P1 VM-Guest" +exit + +# remove existing lag +no interface port-channel 11 + +# create lag for VM Guest +interface port-channel 11 +switchport mode trunk +switchport trunk native vlan 2012 +switchport trunk allowed vlan all +description "SR635 #1 VM Guest LACP" +exit + +# add ports to VM Guest lag in LACP mode +interface Ethernet1/49/3-4 +channel-group 11 mode active +exit + +## SR635 #2 VM Guest + +# configure ports for VM Guest +interface Ethernet1/50/3 +no channel-group +no description +switchport mode trunk +switchport trunk native vlan 2012 +switchport trunk allowed vlan all +description "SR635 #2 S2P1 VM-Guest" +exit + +interface Ethernet1/50/4 +no channel-group +no description +switchport mode trunk +switchport trunk native vlan 2012 +switchport trunk allowed vlan all +description "SR635 #2 S3P1 VM-Guest" +exit + +# remove existing lag +no interface port-channel 12 + +# create lag for VM Guest +interface port-channel 12 +switchport mode trunk +switchport trunk native vlan 2012 +switchport trunk allowed vlan all +description "SR635 #2 VM Guest LACP" +exit + +# add ports to VM Guest lag in LACP mode +interface Ethernet1/50/3-4 +channel-group 12 mode active +exit + +## SR635 #3 VM Guest + +# configure ports for VM Guest +interface Ethernet1/51/3 +no channel-group +no description +switchport mode trunk +switchport trunk native vlan 2012 +switchport trunk allowed vlan all +description "SR635 #3 S2P1 VM-Guest" +exit + +interface Ethernet1/51/4 +no channel-group +no description +switchport mode trunk +switchport trunk native vlan 2012 +switchport trunk allowed vlan all +description "SR635 #3 S3P1 VM-Guest" +exit + +# remove existing lag +no interface port-channel 13 + +# create lag for VM Guest +interface port-channel 13 +switchport mode trunk +switchport trunk native vlan 2012 +switchport trunk allowed vlan all +description "SR635 #3 VM Guest LACP" +exit + +# add ports to VM Guest lag in LACP mode +interface Ethernet1/50/3-4 +channel-group 13 mode active +exit + +#save +show running +exit +copy running-config startup-config +write +``` + +Note: the LACP (channel-group config) setup is being disabled above, only vCentre configured DVswitches may leverage this type of bond. + +## DR site switch + +> ssh admin@129.67.94.7 +> password: Password0 + +``` +enable +configure + +## VM management/vSphere traffic + +# configure ports for management +interface Ethernet1/8-9 +no channel-group +switchport mode trunk +switchport trunk native vlan 2001 +switchport trunk allowed vlan 20,2001,2002 + +## VM iSCSI config + +# configure ports for VM iSCSI +interface Ethernet1/49/1-2 +no channel-group +no description +switchport mode trunk +switchport trunk native vlan 2011 +switchport trunk allowed vlan 2011 +description "SR635 #4 S3P0 iSCSI" +exit + +# client-switch LACP for iSCSI not required + +# remove existing lag +#no interface port-channel 4 + +# create lag for VM iSCSI +#interface port-channel 4 +#switchport mode trunk +#switchport trunk native vlan 2011 +#switchport trunk allowed vlan 2011 +#description "SR635 #4 VM iSCSI LACP" +#exit + +# add ports to VM iSCSI lag in LACP mode +#interface Ethernet1/49/1-2 +#channel-group 4 mode active +#exit + +## VM Guest config + +# configure ports for VM Guest +interface Ethernet1/49/3-4 +no channel-group +no description +switchport mode trunk +switchport trunk native vlan 2012 +switchport trunk allowed vlan all +description "SR635 #4 S3P1 VM-Guest" +exit + +# remove existing lag +no interface port-channel 14 + +# create lag for VM Guest +interface port-channel 14 +switchport mode trunk +switchport trunk native vlan 2012 +switchport trunk allowed vlan all +description "SR635 #4 VM Guest LACP" +exit + +# add ports to VM Guest lag in LACP mode +interface Ethernet1/49/3-4 +channel-group 14 mode active +exit + +# commit +show running +exit +copy running-config startup-config +write +``` + +# Configure ESXi Management networking + +## Configure local web browser + +To configure ESXi hosts via their web interfaces ensure the rdis-20 ssh profile is started and your web browser is configured for a socks5 proxy. + +![4f5062bf662df41e7e48f42cb9675d0b.png](_resources/f9f741fa8af843639cfa429ed52f5695.png) + +## Hypervisor web console login + +> user: root +> pass: Password0 + +| | | +| --- | --- | +| rdis-vmserver01 | https://10.0.1.21 | +| rdis-vmserver02 | https://10.0.1.22 | +| rdis-vmserver03 | https://10.0.1.23 | +| rdis-vmserver04 | https://10.0.1.51 | + +## Setup vSwitch0 nic teaming + +During install the OCP ports are teamed as Active/Failover sets, change these to Active/Active with policy Routed based on originating port ID. Ensure this change is reflected on each ESXi host. + +**![5645875f06763adcb88244c9bb57603f.png](_resources/d8521d3e4c2d4d72bfef9dbb89e9ca12.png)** + +## External management network + +> For vSphere integration a full FQDN / PTR pair are required. + +| | | | +| --- | --- | --- | +| Host | External IP | FQDN / PTR | +| rdis-vmserver01 | 129.67.94.20 | rdis-vmserver01.robots.university.ac.uk | +| rdis-vmserver02 | 129.67.94.21 | rdis-vmserver02.robots.university.ac.uk | +| rdis-vmserver03 | 129.67.94.22 | rdis-vmserver03.robots.university.ac.uk | +| rdis-vmserver04 | 129.67.94.29 | rdis-vmserver04.robots.university.ac.uk | + +| | | +| --- | --- | +| Netmask | 255.255.254.0 /23 | +| Gateway | 129.67.95.254 | +| DNS server | 129.67.1.180 129.67.1.1 | +| DNS suffix | robots.university.ac.uk | +| NTP server | ntp0.robots.university.ac.uk,ntp1.robots.university.ac.uk,ntp2.robots.university.ac.uk,ntp3.robots.university.ac.uk | +| SMTP server | mail.robots.university.ac.uk | + +## Setup port group for external management on VLAN 20 + +![42ed23010245508ce4cc9aee0512a9dd.png](_resources/c2f7ebc0ad2e4d11845fc6b5ec264189.png) + +## Setup VMKernel external interface + +Create a new VMKernel NIC on the External management network port group for each hypervisor. This will likely not be used but assists with initial vCenter installation. + +![49d1560dae5f41b8a7d0d15ccf40a9b3.png](_resources/8ca7fea1dcd24fbda05aea32b3ccb8d9.png) + +## Setup a vMotion portgroup (vlan 2002) + +![54c42d6850afd94f5c709782d17fdb48.png](_resources/527a9ef2d1914702ba5bc13087a9d523.png) + +## Setup VMKernel vMotion interface + +| ESXi host | vMotion IP | +| --- | --- | +| rdis-vmserver01 | 10.0.2.1/24 | +| rdis-vmserver02 | 10.0.2.2/24 | +| rdis-vmserver03 | 10.0.2.3/24 | +| rdis-vmserver04 | Not Required | + +## ![0983031984f26fb1ce5d5190c52f4f7a.png](_resources/8c0878f7a0c74ce7bebd7f11952a7298.png) + +## Rename the default Port Group VM Network + +Port group VM Network is automatically created with vSwitch0 which is in-turn is an uplink trunk to the physical switch with native VLAN 2001. + +This port group is not intended for segregated guest traffic and is in on a non routable IP range utilised for private management interfaces. + +Management VM guests such as XClarity and Think System Manager will have interfaces in this port group (or equivalent DVSwitch port group). + +Rename the port group to VLAN2001_native and set the VLAN ID to 0 to denote native/untagged traffic. + +![29777d87336316a1206e9fb8a6691623.png](_resources/c0387cff72cf45cfa3b822f96dfdbb18.png) + +## Setup hostname, dns and gateway + +![5d2a639ac9b5f9f07eaa2a3f039b999c.png](_resources/478763c554fd4b4ab6ff18c3a79bfc09.png) + +## Setup NTP + +![13fe3554d83a6f38d88b276d2609df2e.png](_resources/321a9036664e48948a1a36be18d3e03f.png) + +Reboot the esxi host, restarting various network services may suffice. + +# License the ESXi hosts + +Whilst this step is duplicated within the vCentre configuration the VCSA appliance installer is buggy on an unlicensed ESXi host. + +| ESXi host | License Key - vSphere v7 (upgraded) | +| --- | --- | +| rdis-vmserver01 | license-key-here | +| rdis-vmserver02 | license-key-here | +| rdis-vmserver03 | license-key-here | +| rdis-vmserver04 | license-key-here | + +## ![f74047a508c64f54945df92803e65559.png](_resources/e9cf53169171448db8297a327bb56055.png) + +# Setup vCenter appliance + +vCenter configuration only applies to rdis-vmserver01-03, rdis-vmserver04 could be managed by vCenter as a stand alone host or a cluster consisting of one machine, however as there is not site based vCentre redundancy and data replication is offloaded between storage arrays I would be a more flexible configuration to manage the host directly. In a scenario where the primary site is offline recovered virtual machines will need to point at differently labelled network and storage to be temporarily brought up on the DR site. + +## Create the VCSA port group + +- This port group is required as the VCSA interface cannot share a management network. +- Create VCSA port group on vlan20. +- For this installation the vCentre is not in HA mode, however to enable recovery/migration of the VCSA appliance ensure the VCSA port group is created on rdis-vmserver01-03. + +![b004e91e9e82833e9cbccc0f63e85388.png](_resources/9aefb3d2311a47588288689373257c2d.png) + +## Install the VCSA appliance + +### Prerequisites + +The VCSA appliance installer will get stuck installing over an SSH forwarded port, call backs from the appliance to itself to install RPM packages hosted on port 443 will use the endpoint address used in the VCSA installer, a spoof local hosts entry is temporarily required be placed on your local laptop and the ssh tunnel run as root to ensure ports <1024 can be mapped locally. + +This is a messy workaround resultant from installation over a chain of ssh tunnels, you could build a virtual machine on the hypervisor for a cleaner installation. + +``` +C:\\Windows\\System32\\drivers\\etc\\hosts +127.0.0.1 rdis-vmserver01.robots.university.ac.uk +``` + +The VCSA appliance has a 365day root password timeout, you may only use ISO files within this release period, download the latest release of the version of VCSA as required. + +The VCSA appliance requires full forward and reverse DNS records. + +### VCSA installer parameters + +| Attribute | Value | +| --- | --- | +| ISO | VMware-VCSA-all-6.7.0-17713310.iso | +| VM name | rdis-vcsa01 | +| Root password | Password0 | +| Deployment size / Storage size | small / default | +| Network | VCSA | +| IP version | IPv4 | +| FQDN | rdis-vcsa01.robots.university.ac.uk | +| IP address | 129.67.94.11 | +| Subnet prefix | 23 | +| Default gateway | 129.67.95.254 | +| DNS servers | 129.67.1.180, 129.67.1.1 | + +### Install the appliance + +- Ensure the local hosts spoof entry is in place for the duration of the installation. +- Run the rdis-vsphere ssh profile as the root user to start a port forward to the ESXi host API endpoint. + +> sudo ssh -F /home/tseed/.ssh/config rdis-vsphere + +- Run the vsphere appliance installer from your local desktop ensuring the target ESXi host is the localhost ssh forwarded port. + +> Download and mount the vsphere VCSA appliance installer ISO +> +> VMware-VCSA-all-6.7.0-17713310.iso +> +> E:\\vcsa-ui-installer\\win32\\installer.exe + +- Select install from the installer, accept any end user agreements, select the embedded controller (applications and databases reside within a single appliance). + +![83ce8e37a833e8819565614903c2165d.png](_resources/ef543104c7e24a5b873c68fd6a053a9f.png) + +Use the ssh port forward endpoint with the spoof hosts entry. + +![4a176fc674b8d681b75632d2e42e10ee.png](_resources/2af89ded3c184ba793a8b6fd93c38648.png) + +Accept the SSL certificate. + +![6703c9ca6bd914f8d52a4244bc139b70.png](_resources/b1b7b264183946ba98a1906ed38172e4.png) + +Configure the name of the virtual machine and VCSA root credentials. + +![400649b3f74f9cac49613bcbebf062f6.png](_resources/3052e6be9e2c4824adccf3180fd629a7.png) + +Select deployment size, the local host datastore is not large enough to thick provision the appliance, ensure thin provisioned disk. + +![a08b640b464cac48885d2b48bb110643.png](_resources/7d59afd025924e96b5f70109c4006e55.png) + +![be62b9d9765c7c1174d16d18c995a202.png](_resources/28e1992d26894a30a05ac41fa436608d.png) + +Configure appliance networking, ensure that the VCSA port group is selected for Network. + +![dfd411751e5e6fc7d9b8aa02340e4422.png](_resources/11701df4a1454b23a672c7d7fb50990c.png) + +The installer will deploy the appliance ready for configuration, make note of the location of the installer log. + +![7b0efac2be3466ee0dc6fa60d0db4659.png](_resources/0da8a4d031bc44009a949b6979a90603.png) + +### Configure the appliance + +- Ensure the rdis-vsphere ssh profile is closed and the temporary hosts file entry is removed. +- Open the rdis-20 ssh profile and continue the installation in a browser using the socks5 profile. + +> https://129.67.94.11:5480 + +Select Set up. + +![f5c6c2f0e6ed63a3f4df6fb36e3575f3.png](_resources/f35c81c82eb04b25904fa15aa4514387.png) + +![5bca8e508ce93afb31d420471fd49fb9.png](_resources/edd2a0f209f6448491c2de239110a049.png) + +Ensure network configuration is correct, enable ssh access and synchronise time with the ESXi host. + +![8f0d42ecd0d23701797b604c7ffa3ed8.png](_resources/6054a228be5e4b68a94f7fa7a1da7ab6.png) + +Configure the mandatory SSO domain. + +| | | +| --- | --- | +| SSO Domain | vcsa.robots.university.ac.uk | +| SSO Admin User | administrator | +| SSO Admin Pass | Password0 | + +![7f5fcb35506ac2098ac3612bc2e53348.png](_resources/57b696f569ec465f872660e21c0e2ee7.png) + +![e32b8e0aa505352a0483354ad02fe05c.png](_resources/e60d37ed62594f3eabe7f8ff7efd5f1a.png) + +Completed. + +![16ad17f8337e4ed5677f348e50149eb4.png](_resources/1cc7db67b44e4b2081ab854878fea7a7.png) + +## vCentre Web consoles + +| Web Console | SSH rdis-20 socks5 address | Internal FQDN | User | Pass | +| --- | --- | --- | --- | --- | +| Appliance administration | https://129.67.94.11:443 | https://rdis-vcsa01.robots.university.ac.uk | administrator@vcsa.robots.university.ac.uk | Password0 | +| Vsphere  administration | https://129.67.94.11:5480 | https://rdis-vcsa01.robots.university.ac.uk:5480 | root | Password0 | + +## vCentre assign licenses + +Ensure the rdis-20 ssh profile is started and your browser is using the local socks5 ssh proxy on 127.0.0.1:8080, edit the local hosts on your laptop to include the following entry: + +> 129.67.94.11 rdis-vcsa01.robots.university.ac.uk + +Login to the appliance administration console with the FQDN rather than the appliance IP and a use compatible browser (google Chrome seems to render the application better than Firefox). + +launch HTML5 portal navigate to menu -> administration -> licensing -> and add new licenses. + +Enter license keys, ensure each key is named with the ESXi host it is being used against. + +| ESXi host | License Key - vSphere v7 | +| --- | --- | +| rdis-vmserver01 | license-key-here | +| rdis-vmserver02 | license-key-here | +| rdis-vmserver03 | license-key-here | +| rdis-vmserver04 | license-key-here | +| rdis-vcsa01 (vCentre license) | license-key-here | + +![9bb6608e27efab2b0c4364e308aade0d.png](_resources/cac0368153944bbf93f2b2211366ecdb.png) + +Navigate to menu -> hosts and clusters then check actions and assign license to vsphere. + +![6b30ae9643a3984a2a1573df6f7af7ec.png](_resources/ce6599d4fa8843e58281772db506f7a2.png) + +## ![95a6f5ed9047db3fda8f9a4c14dfaca6.png](_resources/658d70f211ea4a3f8d10c35fd05fd032.png) + +## Create Datacenter and Cluster + +Navigate to menu -> hosts and clusters, right click on the vcsa object and create a new datacenter named 'Robots'. + +![862e6085f2486537b99df2e7226812f8.png](_resources/bed23312a9984467b998634fe8ac8002.png) + +Right click the Robots datacenter and create a new cluster named 'Primary' with DRS and HA enabled. + +![9627a7278b07d256b6933450d8049689.png](_resources/e225399bab9747a2bfdee180d5cb8840.png) + +![f075f32cdaf353e40e8bfdd56eedbd8d.png](_resources/17d00a30a6364c40b583f05bd06b55d0.png) + +Select the 'Primary' cluster object, navigate to Quickstart, add hosts: + +| Host | External IP | FQDN / PTR | +| --- | --- | --- | +| rdis-vmserver01 | 129.67.94.20 | rdis-vmserver01.robots.university.ac.uk | +| rdis-vmserver02 | 129.67.94.21 | rdis-vmserver02.robots.university.ac.uk | +| rdis-vmserver03 | 129.67.94.22 | rdis-vmserver03.robots.university.ac.uk | +| rdis-vmserver04 (don't add to cluster) | 129.67.94.29 | rdis-vmserver04.robots.university.ac.uk | + +- Ensure each ESXi host is added +- Use credentials root : Password0 +- If asked accept certificate thumbprint and assign the corresponding vSphere license +- If prompted to disable lockdown mode check this option, this can be re-enabled after the cluster is operational and if there is no requirement for direct access to the ESXi host admin console. +- Once the hosts are attached to the Primary cluster object right click each host and check if it has been assigned the corresponding licence, if not assign the correct license. + +![106c3bf8847dcbfd0e65d4af7480e0ce.png](_resources/dacc7e8a722041a1b1a0c620efa0ac48.png) + +Each host will now display multiple errors: + +- Host TPM attestation alarm +- vSphere HA agent on this host could not reach isolation address: 129.67.95.254 +- The number of vSphere HA heartbeat datastores for this host is 0, which is less than required: 2 + +The vSphere HA agent cannot ping the gateway in the RDIS environment, ESXi hosts cannot tell if they are isolated from the network thus HA is disabled, to remedy this we select the switch admin IP as the source of truth for this condition and disable the default gateway check, if a host cannot ping the switch management IP there is a good chance it is isolated. + +> https://kb.vmware.com/s/article/1002478 + +Select 'Primary' cluster, navigate to vSphere Availability -> configure -> vSphere HA is Turned ON -> edit -> advanced options -> add + +![8b3f35cc218ba86fdf809004a32d8e42.png](_resources/2418b30f6166401781c521301330c46b.png) + +To check if this has corrected the fault, disable vSphere HA is Turned ON then re-enable. There should now be a new fault displayed for each hosts summary stating the heartbeat datastore cannot be reached, this will will be cleared once an iSCSI datastore dedicated to heartbeat data is created. + +# Create Management DVSwitch and LACP LAG + +## Remove vmnic5 from vswitch0 on each host in preparation of the DVSwitch creation + +- Select each host in the cluster, navigate to configure -> networking -> virtual switches -> vswitch0 -> manage physical adapters +- Select vmnic5 and remove it from the vswitch + +![a01a2fe783e89752d624a0cd4ef6748d.png](_resources/babba410c963400fa25f61e9744b16e5.png) + +## Create DVSwitch for Management(+vMotion) traffic + +- The DVSwitch will have only 2 uplinks, the ESXi hosts have a total of 6 physical adapters. +- No default port group will be created. +- Network I/O control will be required specifically on the Management DVSwitch for vMotion traffic. +- Ensure the DVSwitch is version 7.0.0 to ensure compatibility with the Lenovo specific ESXi image. + +### Management DVSwitch + +Navigate to networking, right click the Robots datacenter object and add new distributed switch. + +![39815c51d90aac3677cdbac2c854b7c6.png](_resources/e41205193a084698ad3fc28bed7931ff.png) + +![eecce6fa071aacbb372a80819038f8dc.png](_resources/51e71a82980b45f19731a9abb6a0146f.png) + +Select version 7.0.0 to ensure compatibility with the Lenovo specific ESXi image version. + +![660c4cdda55de6ba919af642286d12e8.png](_resources/34965da92ed24ab99aa1472f0ec9b970.png) + +![917b91b661f4e0ad3d1940051bf5a37f.png](_resources/9ad4b039afd047898d6bb94a677a7696.png) + +### Create Management DVSwitch distributed port groups + +All distributed port groups are created with static binding with elastic port allocation. + +Create the following distributed port groups: + +| Distribute port group name | VLAN | +| --- | --- | +| DVP\_VLAN2001\_native | 0 (none) | +| DVP_VCSA | 20 | +| DVP_vMotion | 2002 | +| DVP\_External\_Management_Network | 20 | +| DVP\_Management\_Network | 0 (none) | + +![dd62ae3cefa87bbc03bf3c9028de37c1.png](_resources/69724731f7f14657916850986dfdc4f9.png) + +![e8d086942858c7deb875e0f40dae9de6.png](_resources/022d984329e64e8da11e2385fcae2fd8.png) + +![74045ca682fe2fc63fa04525db8fb96c.png](_resources/a0546f88c8e44d62a1a7196fcc00ebf8.png) + +### Assign the DVSwitch to hosts + +Right click the 'Management' switch, select add and manage hosts. + +![69436fd7382d132f3bef6f0cc661ded5.png](_resources/215fd990336b4659993ec39e09df65ce.png) + +![70e67d938df3498073081607597d80eb.png](_resources/03cf75cfb52a428ab0633a412fd23e0d.png) + +Select physical adapter vmnic5 on each host and assign as uplinks, this is an OCP adapter removed from vSwitch0. + +Assign vmnic5 to uplink2, tick the checkbox to apply the uplink assignment to all of the hosts. + +![044545f5d053734e1137d7e79c23c48e.png](_resources/4915eb00bb6d41958510ecbb0aeb6525.png) + +![64c13a4464aa75335fb26966cf6bae88.png](_resources/e270c7d1342b49a68827e9608fe61268.png) + +The various VMkernel virtual interfaces will need to be migrated to the matching named distributed port groups we have created. + +![c3fb9cf201ed02409032215e8785a4a1.png](_resources/bd5360cbb19b4082a1635ce94c69ba80.png) + +Select each VMkernel vmk0-2, check assign port group, select the matching distributed port group, select apply port assignment for all hosts. + +![9a81439bdb9c38cef9ea67519401eb4a.png](_resources/20ff58b793a94c81ba82ecf9d5928497.png) + +![69b404e039ebb85378b5d53ab7fa980f.png](_resources/f2cc23913119460bb6508c189c288d06.png) + +Existing virtual machines can also be migrated to the distributed port groups, the VCSA appliance must be migrated to maintain connectivity, select the virtual machine and check assign port group, selecting the VCSA distributed port group. The reassigned destination port group will confirm the selection. + +![3cf2648e8988ece42521caa3f5d62540.png](_resources/06dcc26e86ef4660b4f09b9ddb6e8f72.png) + +![1923e9379364a846e0efca79aa224d1f.png](_resources/73013be72b3f458ebeea7d5bc5ab9b43.png) + +You may lose connectivity for a minute or two. + +Remove unused default vSwitch0. + +Navigate to hosts and clusters, select each host, navigate to configure -> networking -> virtual switches and remove vSwitch0. + +![a5cb38537be6b7081fe09fec86d0121f.png](_resources/8bbd442bccb140468b1e3743567b4498.png) + +Add the newly released vmnic4 to the Management DVSwitch on each ESXi host, navigate to the the virtual switches and select manage physical adapters for the DVS_Management switch. + +![91959d696611954815ad1d4c16d05000.png](_resources/7ab1f45dee47408f965d336d5c790c13.png) + +Select Uplink1 and add vmnic4. + +![62e56df3954624b60bf71c7bf769cfdf.png](_resources/46a773bb5e0741b6973fa46af85fe38f.png) + +![19ec43cb027c517f37ff51b605602814.png](_resources/f7725b463d3f4799b4bca44e20fcecf2.png) + +## Create LACP port channel LAG for Management DVSwitch + +The ESXi hosts have a total of 6 physical network adapters, these are equally divided into traffic classes for Management, Storage and Guest for resilience. In this configuration the vMotion traffic will traverse the Management network adapters, to maximise single socket bandwidth these ports will be moved to a LACP LAG. + +Generally it is preferable to have a dedicated standard vSwitch physical adapter not in a LAG and not on a DVSwitch for easy access when troubleshooting. Physical adapter restraints and the fact that this cluster is unlikely to have many network topology changes over its lifecycle dictate a preference for maximum bandwidth specifically for the vMotion traffic thus setting the topology of the Management physical adapters for LACP. + +At this stage the Management DVSwitch has vmnic4-5 set as trunk ports, vmnic4 will be moved to a LAG with complimentary physical switch configurations updated and a migration of DVSwitch traffic to this LAG, once this is complete vmnic5 will be moved to the LAG to complete the operation. + +### Remove vmnic4 from the DVS_Management virtual switch + +From the vCentre web client navigate to hosts and clusters, select each ESXi host -> configure -> networking -> virtual switches, select DVS_Management switch -> manage physical adapters. + +Select Uplink1/vmnic4 and remove. + +![44dd06388f001cee5ef67c17cd52f467.png](_resources/23d3e7d8b3da41e6a984a75f76603066.png) + +### Create the Management LAG + +From the vCentre web client navigate to networking, select the DVS_Management switch, select configure -> LACP -> NEW. + +Create a new LAG named 'LAG_Management' with 2 ports in passive mode with load balancing profile 'source and destination IP address, TCP/UDP port and VLAN' this will loadbalance and aggregate across layer 3 traffic, generally layer 2 MAC address policies aren't required over virtual networks. + +![a74cafc819f9e9106e369bd7be7ebb91.png](_resources/be1faac136bb4960a932838a602e4619.png) + +Select migrating network traffic to LAGs then manage distributed port groups in the popup. + +## ![7095b4e733456644c2b516df84922e6d.png](_resources/35fa79a1019c41e791d621804eebb6fd.png) + +Check teaming and failover. + +![1fd5cc6eae24899f0f61eb89a0a71aed.png](_resources/255bfbb9bf2a4ddcb5dc2edae6f9483c.png) + +Select all of the management distribute port groups. + +| Distributed port groups | +| --- | +| DVP\_External\_Management_Network | +| DVP\_Management\_Network | +| DVP_VCSA | +| DVP\_VLAN2001\_native | +| DVP_vMotion | + +![11dda204c8ee5938844009ec383c7afc.png](_resources/54c0314cb4374aac8196ea0b74d48813.png) + +Move the LAG 'LAG_Management' into the standby uplink list and accept all teaming defaults, the LAG cannot become an active uplink until the physical switch is configured for LACP port channel, this will show a warning stating this is an 'invalid' intermediate configuration intended for migration to the LAG, this is as expected. + +![e2e9fda093d374310839b213bedaaa43.png](_resources/968b35a674c84fa89992b141ef598eb1.png) + +![1e04656c8f984bfc91776ca758fec9b4.png](_resources/930a6d6646814e279ce52b3350caf94d.png) + +Select migrating network traffic to LAGs then add and manage hosts in the popup. + +![bb65b181dab19dda139c308aed452c3c.png](_resources/8a6a3e4af5fb44208f53c8db64c1a79b.png) + +Select manage host networking + +## ![cb3eca5c471bad16bf9053ff3623fe08.png](_resources/81e80982157f41a896d5ae0ba194c304.png) + +Select all hosts in the 'Primary' cluster. + +![5ebbdb356598f31e530ad14cae903d30.png](_resources/12b5a9b56e9e4e299bccd8e137f2bc25.png) + +Assign vmnic4 to LAG_Management-0 port, apply this uplink to all hosts. + +![47161cd6ad3b61d178aba0ead4c727fd.png](_resources/d8873683b955480cbcdaa445b5f2e0e4.png) + +Do not migrate any VMKernel adapters or virtual machines to any port groups at this stage. The topology will now show the management lag containing vmnic4 for each ESXi host. + +![e45132c2aea9ce365b776bbbb0e1047f.png](_resources/295c9b73b80c4680814d416034174036.png) + +## Configure the LAG port channels on the primary site switch. + +> ssh admin@129.67.94.5 +> password: Password0 + +``` +## VM Management traffic LACP for vmnic4 + +enable +configure + +## SR635 #1 VM Management + +# remove existing lag +no interface port-channel1 + +# create lag for VM Management +interface port-channel1 +switchport mode trunk +switchport trunk native vlan 2001 +switchport trunk allowed vlan 2001,2002,20 +description "SR635 #1 VM Management LACP" +exit + +#configure vmnic4 port for VM Management +interface Ethernet1/16 +channel-group 1 mode active +exit + +## SR635 #2 VM Management + +# remove existing lag +no interface port-channel2 + +# create lag for VM Management +interface port-channel2 +switchport mode trunk +switchport trunk native vlan 2001 +switchport trunk allowed vlan 2001,2002,20 +description "SR635 #2 VM Management LACP" +exit + +#configure vmnic4 port for VM Management +interface Ethernet1/18 +channel-group 2 mode active +exit + +## SR635 #3 VM Management + +# remove existing lag +no interface port-channel3 + +# create lag for VM Management +interface port-channel3 +switchport mode trunk +switchport trunk native vlan 2001 +switchport trunk allowed vlan 2001,2002,20 +description "SR635 #3 VM Management LACP" +exit + +#configure vmnic4 port for VM Management +interface Ethernet1/20 +channel-group 3 mode active +exit + +show running +exit +copy running-config startup-config +write +``` + +### Set the now active LAGs to be active for the distributed port groups and remove the native DVswitch uplink (vmnic5 adapter) + +Select migrating network traffic to LAGs then manage distributed port groups in the popup. + +![0efb7bca68218394f032bb9361cf0cfb.png](_resources/62cf7bde6921423b841f2ac6f64f76e6.png) + +Select traffic teaming and failover. + +![1fd5cc6eae24899f0f61eb89a0a71aed.png](_resources/255bfbb9bf2a4ddcb5dc2edae6f9483c.png) + +Select all of the management distributed port groups. + +| | +| --- | +| DVP\_External\_Management_Network | +| DVP\_Management\_Network | +| DVP_VCSA | +| DVP\_VLAN2001\_native | +| DVP_vMotion | + +![11dda204c8ee5938844009ec383c7afc.png](_resources/54c0314cb4374aac8196ea0b74d48813.png) + +Move Uplink1-2 to unused uplink list and move LAG_Management to active uplinks. + +![ea8533f529f95fee0ce43667d61d0386.png](_resources/6224bcfaac054e01b1987447cd9d3684.png) + +Refresh the topolgy page and check that traffic is now only active over the LAG Management uplink. + +![4657ff5b8d0ab007849bf7de2d78ed3a.png](_resources/0c969f08ddfa4d2a9f9eb73b74f1db7f.png) + +## Remove vmnic5 from DVS\_Management switch on each host in preparation to add to the LAG LAG\_Management + +- Select each host in the cluster, navigate to configure -> networking -> virtual switches -> DVS_Management -> manage physical adapters +- Select Uplink2 vmnic5 and remove + +![1595153710c82655028484e9bca6ba9f.png](_resources/51e469ed017140a2b3f3df2e84ba2396.png) + +## Configure the LAG port channels on the primary site switch to include the now released vmnic5 + +> ssh admin@129.67.94.5 +> password: Password0 + +``` +## VM Management traffic LACP for vmnic5 + +enable +configure + +## SR635 #1 VM Management + +#configure vmnic5 port for VM Management +interface Ethernet1/17 +channel-group 1 mode active +exit + +## SR635 #2 VM Management + +#configure vmnic5 port for VM Management +interface Ethernet1/19 +channel-group 2 mode active +exit + +## SR635 #3 VM Management + +#configure vmnic5 port for VM Management +interface Ethernet1/21 +channel-group 3 mode active +exit + +show running +exit +copy running-config startup-config +write +``` + +## Add vmnic5 to the DVS\_Management switch uplink LAG LAG\_Management + +From the vCentre web client navigate to networking, select the DVS_Management switch, select configure -> LACP -> migrating network traffic to LAGs -> add and manage hosts. + +![0efb7bca68218394f032bb9361cf0cfb.png](_resources/62cf7bde6921423b841f2ac6f64f76e6.png) + +Select manage host networking. + +![102759762e8244e1f1004dd27d0c4072.png](_resources/fa03562970de47b1a1dc55296243293c.png) + +Select ESXi hosts. + +![8fa6a84459bdbcc88a0c12bb9c2f54b0.png](_resources/f48ec38a008c42c78d775a2e5b90e163.png) + +Select vmnic5 and assign uplink to LAG_Management-1 and apply to all hosts. + +![cb5f6321e9c4f39b067fa21655dfb7a4.png](_resources/af7371d134bb4ba490308c984bc96754.png) + +Do not migrate any VMKernel adapters or virtual machines to any port groups at this stage. + +The topology will now show active traffic on the vmnic5 interface. + +![e7b22a966ece0bbc4db4c01b3653b993.png](_resources/da77efd4b6904afeb143a98bcfd4ee25.png) + +The switch console should show the LACP port channel up and the vmnic5 adapters leave the suspended state. + +![6d69b16868bcf9a461c593bd864337bd.png](_resources/10b094b022664912b2bd775628b7f06f.png) + +# Create Guest LACP LAG and DVSwitch + +## Create DVSwitch for Guest traffic + +- The DVSwitch will have only 2 uplinks, the ESXi hosts have a total of 6 physical adapters. +- No default port group will be created. +- Ensure the DVSwitch is version 7.0.0 to ensure compatibility with the Lenovo specific ESXi image. + +### Guest DVSwitch + +Navigate to networking, right click the Robots datacenter object and add new distributed switch. + +![39815c51d90aac3677cdbac2c854b7c6.png](_resources/e41205193a084698ad3fc28bed7931ff.png) + +![f6b7e9a990d3ed42d91d968ca6045e9d.png](_resources/dd25e93ac646497b80518e34af7227d1.png) + +Select version 7.0.0 to ensure compatibility with the Lenovo specific ESXi image version. + +![660c4cdda55de6ba919af642286d12e8.png](_resources/34965da92ed24ab99aa1472f0ec9b970.png) + +![917b91b661f4e0ad3d1940051bf5a37f.png](_resources/9ad4b039afd047898d6bb94a677a7696.png) + +## Create Guest LAG + +From the vCentre web client navigate to networking, select the DVS_Guest switch, select configure -> LACP -> NEW. + +Create a new LAG named 'LAG_Guest' with 2 ports in passive mode with load balancing profile 'source and destination IP address, TCP/UDP port and VLAN' this will loadbalance and aggregate across layer 3 traffic, generally layer 2 MAC address policies aren't required over virtual networks. + +Select the new LAG then migrating network traffic to LAGs ->  add and manage hosts -> add hosts. + +![844e7140ce3a9ef7bca7c4ad73ca82b7.png](_resources/b0943b689efe4084bca3621451f300e2.png) + +Select all hosts in the 'Primary' cluster. + +## ![148345388ae9880778be9ff90a5edc74.png](_resources/c724a19cda234daeafa842ea89c59824.png) + +Select the vmnic adapters and assign as uplinks for all ESXi hosts. + +- The initial physical switch configuration has the VM Guest LACP port channel configured. +- The supporting info section of this document lists the vmnic adapters and their corresponding physical switch port ID. +- For quick reference use the vmnic1 and vmnic3 adapters as LAG\_Guest-0 and LAG\_guest-1 for the DVS_Guest LACP configuration. + +![f62b568f5a0f87406520223f1c81f360.png](_resources/3bf8cc2363214be3941bc16883aba061.png) + +## ![dbb2f59416ca1208ddd58525267fc59d.png](_resources/fe41ea383f3749d08eee3202814774bc.png) + +Do not migrate any VMKernel adapters or virtual machines to any port groups at this stage. + +The switch console will show messages relating to port channel 11-13 negotiating and being pinned up, check the switch output to confirm: + +![eae656ba335e726c10e804b886ba1805.png](_resources/30de07198d74431089f1212812d1e289.png) + +The topology should show LAG_guest with two active vmnics per ESXi host. + +![3ceafb30fa12817e5aa27a8ee116b0af.png](_resources/38caa0982c2546baaeec64220fecefd5.png) + +- To home virtual machines on this guest DVSwitch, distributed port groups must be created with an appropriate VLAN tag presented across the uplinks from the customer core switches. +- The physical trunk ports on the switch are set with a native VLAN of 2001 with no VLAN mask, 0-4096 allowed. + +# Create Storage DVSwitch + +## Set the physical switch port to access mode on VLAN 2011 + +### Primary site switch + +> ssh admin@129.67.94.5 +> password: Password0 + +```sh +enable +configure + +interface Ethernet1/49/1 +description "SR635 #1 S2P0 iSCSI" +switchport mode access +switchport access vlan 2011 +exit + +interface Ethernet1/49/2 +description "SR635 #1 S3P0 iSCSI" +switchport mode access +switchport access vlan 2011 +exit + +interface Ethernet1/50/1 +description "SR635 #2 S2P0 iSCSI" +switchport mode access +switchport access vlan 2011 +exit + +interface Ethernet1/50/2 +description "SR635 #2 S3P0 iSCSI" +switchport mode access +switchport access vlan 2011 +exit + +interface Ethernet1/51/1 +description "SR635 #3 S2P0 iSCSI" +switchport mode access +switchport access vlan 2011 +exit + +interface Ethernet1/51/2 +description "SR635 #3 S3P0 iSCSI" +switchport mode access +switchport access vlan 2011 +exit + +show running +exit +copy running-config startup-config +write +``` + +### DR site switch + +> ssh admin@129.67.94.7 +> password: Password0 + +```sh +enable +configure + +interface Ethernet1/49/2 +description "SR635 #4 S3P0 iSCSI" +switchport mode access +switchport access vlan 2011 +exit + +interface Ethernet1/49/3 +description "SR635 #4 S3P1 iSCSI" +switchport mode access +switchport access vlan 2011 + +show running +exit +copy running-config startup-config +write +``` + +## Create DVSwitch for Storage traffic + +- The DVSwitch will have only 2 uplinks, the ESXi hosts have a total of 6 physical adapters. +- No default port group will be created. +- Ensure the DVSwitch is version 7.0.0 to ensure compatibility with the Lenovo specific ESXi image. +- No port/channel bonding is in use, the storage vendor recommends iSCSI connectivity is multipath'd without LACP at both the storage and client endpoints. + +### Storage DVSwitch + +Navigate to networking, right click the Robots datacenter object and add new distributed switch. + +![39815c51d90aac3677cdbac2c854b7c6.png](_resources/e41205193a084698ad3fc28bed7931ff.png) + +![23e65d16a6e80395d1f37a67b4fb5121.png](_resources/518bb25d1df54c5f81d8ad1ef3996acd.png) + +Add hosts to the switch. + +![d136d5d14ece63462206eff4eb7f7cf2.png](_resources/f62b897723f24f6483f92c145f266f56.png) + +![aa894f24ca4535c31cdb55e14b68a735.png](_resources/8bacf6fbee4546dc848ef9554e089bff.png) + +Select all hosts in the 'Primary' cluster. + +![c9b6f987abd6e2c54f63ac8976053dbf.png](_resources/b38e5e7b436f454b9fc6c55f649658a0.png) + +Select the vmnic adapters and assign as uplinks for all ESXi hosts. + +- The initial physical switch configuration has the VM Storage ports configured as trunks. +- The supporting info section of this document lists the vmnic adapters and their corresponding physical switch port ID. +- For quick reference use the vmnic0 and vmnic2 adapters as LAG\_Guest-0 and LAG\_guest-1 for the DVS_Guest LACP configuration. + +![aa7c2b1686a6706a4af3349bf9e13b1f.png](_resources/a590de9e6f3f4894bd9f7b945fefe415.png) + +![1f8b3e9b590621392c5855df9a0b3481.png](_resources/fa2fc8009ec9431da2c3d41097467540.png) + +![ab580e1526dbe301c709223b3a9a6408.png](_resources/1c68eaf4fef34f698abaf1e696199bb0.png) + +## Create distributed port groups for the iSCSI targets. + +| Port group | vSwitch | +| --- | --- | +| DVP\_ISCSI\_A | DVS_Storage | +| DVP\_ISCSI\_B | DVS_Storage | +| DVP\_ISCSI\_C | DVS_Storage | +| DVP\_ISCSI\_D | DVS_Storage | + +Navigate to networking, expand the Robots datacenter object, open the context menu for switch DVS_Storage and add the listed distributed port groups. + +Select defaults for the port group and ensure VLAN is set to none, the trunk native VLAN is 2011, the storage subnet is supernetted containing /27 ranges. + +![4bac0fda5ff7dc0f5767cd97693ce43b.png](_resources/a0b9cc8ba9b34a3495b5645edc931fca.png) + +![a700d34b27d0ed8014ed0768f50c1f12.png](_resources/7481d5c09d27462ba7fc877d27839270.png) + +![8a63adbe92da40b5e0f8d02e98c4b771.png](_resources/80cdc5332e52444e849da2831e9e1584.png) + +![be62967071cfb66321fa3d04cb44ad6c.png](_resources/ded1674d6c3e43cf98e4ac057c779181.png) + +# Configure the DR cluster + +- The DR cluster will consist of a single ESXI host, rdis-vmserver04. +- A cluster is used in preference to a standalone host to enable vmotion between clusters and keep the distributed switch architecture. +- An emergency / full DR is a manual process requiring a the replicated volumes to be promoted from (read-only) secondary status to (read-write) primary status via the storage array controller admin interfaces. +- A planned DR / maintenance event - where essential virtual machines can vmotion from one cluster to another without a requirement to promote storage, can occur as a live migration with this configuration. + +## Create new cluster + +From the vCentre admin console navigate to hosts and clusters, select the Robots datacenter and create new cluster. + +![268f1df34707ba0f6b2d7aa53158abe3.png](_resources/d4445738410045ab86a6637f949f1a3a.png) + +Uncheck DRS, HA and vSAN, this cluster will consist of a single host. + +![df9a1f3e4412ff7a4c3041c7487f6211.png](_resources/c62d133dbe7442a2878ce634876c8658.png) + +Select the 'DR' cluster and open the context menu, add hosts, add rdis-vmserver04. + +![69e438c9078966730ca9f5507d2202ab.png](_resources/deec045da79e4f6ca7f4e55b71ddd808.png) + +## Configure the Management network for rdis-vmserver04 + +### Release vmnic5 from vswitch0 + +Select rdis-vmserver04, navigate to configure -> networking -> virtual switches -> vswitch0 -> manage physical adapters + +Select vmnic5 and remove it from the vswitch0. + +![ca97879c7e3c429989e512ea6340df4f.png](_resources/aa71851f6bb0447db8fe8aef71fe42f6.png) + +![6094ea5679742ed698e702f136fcb085.png](_resources/42eff0119677431c95cf8a1a9283ca05.png) + +Assign rdis-vmserver04 to the DVS_Management switch. + +![f0789c6f7566ef7f781c551b802a8094.png](_resources/7156faca53764780a2fe914c7668534a.png) + +![24b66b6677a43375a2c43234baaa5ea7.png](_resources/5605fa906855401e833663f2d05d669f.png) + +Do not assign any vmnic adapters from the host to the distributed switch at this stage. The DVP_Management switch is configured as a LACP lag, vmnic adapters must be assigned to the lag not the switch. + +Navigate to hosts and clusters, select the rdis-vmserver04 host in the 'DR' cluster, select configure -> virtual switches -> DVS_Management -> manage physical adapters. + +Select the uplink port named LAG_Management-1 and attach vmnic5. + +### Configure the LAG port channel on the DR site switch to include vmnic5 + +Login to the DR site switch and configure the vmnic5 physical switch port to be in a LACP port channel. + +ssh admin@129.67.94.7 + +password: Password0 + +Move vmnic5 to the port channel lag. + +```sh +## VM Management traffic LACP for vmnic5 + +enable +configure + +## SR635 #4 VM Management + +# remove existing lag +no interface port-channel4 + +# create lag for VM Management +interface port-channel4 +switchport mode trunk +switchport trunk native vlan 2001 +switchport trunk allowed vlan 2001,2002,20 +description "SR635 #4 VM Management LACP" +exit + +#configure vmnic5 port for VM Management +interface Ethernet1/9 +channel-group 4 mode active +exit + +show running +exit +copy running-config startup-config +write +``` + +Navigate to hosts and clusters, select the rdis-vmserver04 host in the 'DR' cluster, select configure -> virtual switches -> DVS_Management -> migrate networking. + +Do not select any vmnic adapters to be assigned to an uplink. + +Select VMkernel0 and VMkernel1 and assign to distributed port groups, DVP\_Management\_Network and DVP\_External\_Management_Network respectively. + +![d703e8759470a152dfb553aefc5b7fa2.png](_resources/339d1d64f5aa437aa31a11ee78dcfdb1.png) + +Wait for the ESXi host to be available, this should be no longer than a minute, check you can connect to the hosts admin console to confirm changes https://129.67.94.29/ui. + +### Remove the standard vSwitch0 virtual switch on rdis-vmserver04 + +Navigate to hosts and clusters, select the rdis-vmserver04 host in the 'DR' cluster, select configure -> virtual switches -> vSwitch0 -> remove. + +the vmnic4 adapter will now be released. + +### Add vmnic4 to the DVS_Management distributed virtual switch LAG uplink + +Select the DVS\_management switch, check manage physical adapters, select LAG\_Management-0 and assign vmnic4 + +### Configure the LAG port channel on the DR site switch to include vmnic4 + +Login to the DR site switch and configure the vmnic5 physical switch port to be in a LACP port channel. + +ssh admin@129.67.94.7 + +password: Password0 + +Move vmnic4 to the port channel lag. + +```sh +## VM Management traffic LACP for vmnic4 + +enable +configure + +## SR635 #4 VM Management + +#configure vmnic4 port for VM Management +interface Ethernet1/8 +channel-group 4 mode active +exit + +show running +exit +copy running-config startup-config +write +``` + +Check port channel 4 is up. Channel-group 14 has an issue in this example. + +![370e2e9b611fe8120f9e6339be7d2e49.png](_resources/2669a3466b30494a827b3b12fa222f12.png) + +### Create vMotion VM kernel interface for rdis-vmserver04 + +Navigate to hosts and clusters, select the rdis-vmserver04 ESXi host in the 'DR' cluster, select configure -> VMkernel adapters -> add networking. + +Select VMkernel, select existing network 'DVP_vMotion', check enabled services 'vMotion'. This should be VMkernel device vmk2 at this stage of the build. + +Set a static IPv4 address 10.0.2.21/24. + +![89cb4db87ccaf90e79e37f9db2d2e66e.png](_resources/5a9e800f004542c1976f94e87250b550.png) + +## Configure the Guest network for rdis-vmserver04 + +Navigate to networking, select the DVS_Guest distributed virtual switch from the Robots datacenter, open the context menu -> add and manage hosts, select rdis-vmserver04. + +Do not add any vmnic adapters, these will be added to the LACP uplinks, do not migrate any VMkernels or VM networking (distributed port groups). + +Navigate to hosts and clusters, select the rdis-vmserver04 ESXi host in the 'DR' cluster, select configure -> virtual switches -> DVS_guest -> manage physical adapters. + +Select LAG\_Guest-0 add adapter vmnic1, select LAG\_Guest-1 add adapter vmnic3 + +![7b9eef36f5bf540e23ee78470a53dc40.png](_resources/fd37e729ea804d939c4d56e099675433.png) + +## Configure the Storage network for rdis-vmserver04 + +Navigate to networking, select the DVS_Storage distributed virtual switch from the Robots datacenter, open the context menu -> add and manage hosts, select rdis-vmserver04. + +Add vmnic adapters vmnic0 and vmnic2 to uplinks 1 and 2 respectively. There are no LACP uplinks used for storage. + +Do not migrate any VMkernels or VM networking (distributed port groups). + +# Configure Storage + +## Create the VMkernel interfaces for each server to access the iSCSI multipath target ports. + +| Server | Portgroup | VMkernel | IP | +| --- | --- | --- | --- | +| rdis-vmserver01 | DVP\_iSCSI\_A | vmk3 | 10.0.3.11/27 | +| rdis-vmserver01 | DVP\_iSCSI\_B | vmk4 | 10.0.3.43/27 | +| rdis-vmserver01 | DVP\_iSCSI\_C | vmk5 | 10.0.3.75/27 | +| rdis-vmserver01 | DVP\_iSCSI\_D | vmk6 | 10.0.3.107/27 | +| rdis-vmserver02 | DVP\_iSCSI\_A | vmk3 | 10.0.3.13/27 | +| rdis-vmserver02 | DVP\_iSCSI\_B | vmk4 | 10.0.3.45/27 | +| rdis-vmserver02 | DVP\_iSCSI\_C | vmk5 | 10.0.3.77/27 | +| rdis-vmserver02 | DVP\_iSCSI\_D | vmk6 | 10.0.3.109/27 | +| rdis-vmserver03 | DVP\_iSCSI\_A | vmk3 | 10.0.3.15/27 | +| rdis-vmserver03 | DVP\_iSCSI\_B | vmk4 | 10.0.3.47/27 | +| rdis-vmserver03 | DVP\_iSCSI\_C | vmk5 | 10.0.3.79/27 | +| rdis-vmserver03 | DVP\_iSCSI\_D | vmk6 | 10.0.3.111/27 | +| rdis-vmserver04 | DVP\_iSCSI\_A | vmk3 | 10.0.3.17/27 | +| rdis-vmserver04 | DVP\_iSCSI\_B | vmk4 | 10.0.3.49/27 | +| rdis-vmserver04 | DVP\_iSCSI\_C | vmk5 | 10.0.3.81/27 | +| rdis-vmserver04 | DVP\_iSCSI\_D | vmk6 | 10.0.3.113/27 | + +Navigate to hosts and clusters in the vCentre admin console, for each ESXi host select configure -> networking -> vmkernel adapters -> add networking -> vmkernel network adapter + +![c0ba430c285ebe0727c3c21aa9583b0c.png](_resources/36e4bf32ab6545a38b9a93d25cc76665.png) + +Browse and select the distributed port group, accept defaults and do not enable specific services (management, vmotion etc). + +![82769350c7978c4217fca3a36081c170.png](_resources/ae4ae87c717342d7af0ecd2852f17aeb.png) + +Configure the IP address for the VMkernel interface. + +![5d0a74d3e884ee82c2dd0a502fcbb5f2.png](_resources/fbc952d2e7ca4ad48eb3cbeed53a038b.png) + +A host with a full compliment of iSCSI VMKernel adapters will resemble. + +![884dac0546dd91adaa838bfd7f6a43eb.png](_resources/f8ad1ce9677b420ba21268f53423dd4c.png) + +To continue iSCSI target configuration, the storage array networking requires adjustment for multipath. + +# Configure Storage Array + +Configure the Primary and DR site storage arrays. + +### Links + +https://thinksystem.lenovofiles.com/storage/help/index.jsp + +Management Software: + +- ThinkSystem System Manager - embedded web-based management console on each controller +- Secure Command Line Interface - CLI +- ThinkSystem SAN Manager - Web application installed on a server instance configured with a web server with proxy configuration + +## Array web console login + +> user: admin +> pass: Password0 + +| Site | Array | Controller consoles | +| --- | --- | --- | +| Primary | DE2000H | https://10.0.1.1:8443
https://10.0.1.2:8443 | +| Primary | DE4000H | https://10.0.1.3:8443
https://10.0.1.4:8443 | +| DR | DE4000H | https://10.0.1.31:8443
[https://10.0.1.32:8443](https://10.0.1.31:8443) | + +### Array Storage Management Interface Configuration + +Ensure interfaces are configured with the following IP attributes. + +| Site | Array | Controller | Management Port | Label | IP | VLAN
(native/untagged) | +| --- | --- | --- | --- | --- | --- | --- | +| Primary | DE2000H | A | P1 (P2 reserved by vendor) | rdis-vmshelf01a | 10.0.1.1/24 | 2001 native | +| Primary | DE2000H | B | P1 (P2 reserved by vendor) | rdis-vmshelf01b | 10.0.1.2/24 | 2001 native | +| Primary | DE4000H | A | P1 (P2 reserved by vendor) | rdis-vmshelf02a | 10.0.1.3/24 | 2001 native | +| Primary | DE4000H | B | P1 (P2 reserved by vendor) | rdis-vmshelf02b | 10.0.1.4/24 | 2001 native | +| DR | DE4000H | A | P1 (P2 reserved by vendor) | rdis-vmshelf03a | 10.0.1.31/24 | 2001 native | +| DR | DE4000H | B | P1 (P2 reserved by vendor) | rdis-vmshelf03b | 10.0.1.32/24 | 2001 native | + +### Array Storage iSCSI Interface Configuration + +Multipath IP configuration scheme example: + +[https://thinksystem.lenovofiles.com/storage/help/index.jsp?topic=%2Fthinksystem\_storage\_de\_himg\_11.60.2%2FConfigure\_hosts\_Windows.html](https://thinksystem.lenovofiles.com/storage/help/index.jsp?topic=%2Fthinksystem_storage_de_himg_11.60.2%2FConfigure_hosts_Windows.html) + +Ensure interfaces are configured with the following IP scheme. + +| Site | Array | Controller | Host Interface | Label | IP | VLAN | Subnet | +| --- | --- | --- | --- | --- | --- | --- | --- | +| Primary | DE2000H | A | e0c | rdis-vmshelf01a | 10.0.3.1/27 | 2011 native | A | +| Primary | DE2000H | A | e0d | rdis-vmshelf01a | 10.0.3.33/27 | 2011 native | B | +| Primary | DE2000H | B | e0c | rdis-vmshelf01b | 10.0.3.65/27 | 2011 native | C | +| Primary | DE2000H | B | e0d | rdis-vmshelf01b | 10.0.3.97/27 | 2011 native | D | +| Primary | DE4000H | A | e0c | rdis-vmshelf02a | 10.0.3.2/27 | 2011 native | A | +| DR | DE4000H | A | e0d | rdis-vmshelf02a | 10.0.3.34/27 | 2011 native | B | +| DR | DE4000H | B | e0c | rdis-vmshelf02b | 10.0.3.66/27 | 2011 native | C | +| DR | DE4000H | B | e0d | rdis-vmshelf02b | 10.0.3.98/27 | 2011 native | D | +| DR | DE4000H | A | e0c | rdis-vmshelf02a | 10.0.3.3/27 | 2011 native | A | +| DR | DE4000H | A | e0d | rdis-vmshelf02a | 10.0.3.35/27 | 2011 native | B | +| DR | DE4000H | B | e0c | rdis-vmshelf02b | 10.0.3.67/27 | 2011 native | C | +| DR | DE4000H | B | e0d | rdis-vmshelf02b | 10.0.3.99/27 | 2011 native | D | + +## Disable unused network adapters in the storage array heads + +The purpose of this is to ensure the non routable address ranges are not displayed in iSCSI discovery with VMWare. + +Login to each array and disable the unused interfaces each controller. + +Navigate to settings -> system -> configure iSCSI ports -> Controller A -> Port 0a -> disable IPv4 / IPv6 + +Navigate to settings -> system -> configure iSCSI ports -> Controller A -> Port 0b -> disable IPv4 / IPv6 + +Navigate to settings -> system -> configure iSCSI ports -> Controller B -> Port 0a -> disable IPv4 / IPv6 + +Navigate to settings -> system -> configure iSCSI ports -> Controller B -> Port 0b -> disable IPv4 / IPv6 + +## Array Disk Pool Configuration + +All disk in the DE2000H and DE4000H arrays are configured in a single DDP (dynamic disk pool) pool, Pool0. + +# ESXi Create iSCSI storage adapters + +Navigate to hosts and clusters and select each ESXi host in the 'Primary' cluster and perform the following task. + +Select configure -> storage -> storage adapters -> add storage adapter -> add software iSCSI adapter + +![824cdd06ea827c58470806cd901f289e.png](_resources/a90a5a623f2e43a8855329b4cb768c25.png) + +![aa7bbb9f8875e0d0b995e447e201bb31.png](_resources/4dd2b7273b3b490aa2b99ad464ee844f.png) + +The iSCSI adapter will not use port bindings, chap authentication (isolated storage network) or any non default configuration. + +For the ESXi hosts to discover iSCSI exports an ACL must be populated on each storage array. + +## Create a host object for each ESXi host on each storage array + +- This acts an ACL to allow granular access for sets of ESXi hosts. +- The host IQN identifier can be found in the vCenter console at the iSCSI adapter for each host. +- Add each ESXi host rdis-vmserver01-04 as a host profile on each storage array. + ![692f472c2ead4a4f3c94643a7926d7d4.png](_resources/1644bc1070a4482795866192c3126190.png) + +![73fc64844c1dbc4dace426d7d33512b8.png](_resources/bc8367c6731e4dd1b7acaf703361a4f9.png) + +![fcd7fae2d39ff1fed6a64a3804419c2a.png](_resources/2d505be2d62d4b2e98795d4103fa6239.png) + +### Create a host cluster + +A volume can only be assigned to one host object, to allow the DR host rdis-vmserver04 to access volumes on the primary site storage arrays add all hosts to the rdis-vmserver cluster object. + +![4238cabf31d7afb1b47b8bccd7feba07.png](_resources/c70235ca5a094c249ad28b7c33c4d960.png) + +## Dynamic discovery for iSCSI endpoints + +Once the iSCSI adapter is present on each ESXi host and the each storage array has a populated iSCSI ACL, perform a dynamic discovery + +For each ESXi host navigate to hosts and clusters -> configure -> storage adapters, select the iSCSI adapter -> dynamic discovery. + +Enter an IP for each storage array, the discovery process will enumerate all other endpoints on the storage in the A/B/C/D subnets, enter addresses 10.0.3.1, 10.0.3.2 and 1.0.3.3. + +![66c280548d88420b9d4519866ab217df.png](_resources/8c8561c02bd044eaba9136fa966dab4e.png) + +Select rescan adapter. + +![2daa066695286d466d071793491f3851.png](_resources/c1d5eeb76e0140c4b8c4d401b4cff13c.png) + +- The Static targets tab will now be populated. Even though only 1 IP of the 4 used on each storage array was used for discovery, all iSCSI IP endpoints are discovered for each array. +- The target IQN endpoint for an array remains the same regardless of the controller or physical adapter IP/endpoint, there is now a multipath circuit to each array, on each array controller and each array controllers configured physical network adapters. +- If IQNs are listed with 192.168.130/131 addresses (or any inaccessible IPs), these are likely the default IP's for unconfigured physical adapters on the array controllers (usually 1Gbps links), these can be removed from the static target list but it is preferable to disable the IP stack for this adapter on the storage array controllers. + +![0d0250c09ce1295e0a64e2ef0a6881c1.png](_resources/651bc4bac2ef4ad19d2647174dff9747.png) + +Rescan the iSCSI adapter to remove configuration change notifications. + +# Create test volumes on each storage array + +Refer to this table to name storage volumes and vCenter datastores. + +| Storage Array | Volume Name (VMware datastore name) | Workload Name | Assign Host Cluster | Note | +| --- | --- | --- | --- | --- | +| rdis-vmshelf01 | PRI\_DE2000H\_vmcluster_admin | PRI\_DE2000H\_vmfs | rdis-vmserver | admin volume mounted as datastore on Primary and DR clusters, this should house essential virtual machines such as the VCSA appliance. | +| rdis-vmshelf01 | PRI\_DE2000H\_vol | PRI\_DE2000H\_vmfs | rdis-vmserver | example volume to be mounted as datastore and replicated to DR array | +| rdis-vmshelf02 | PRI\_DE4000H\_vol | PRI\_DE4000H\_vmfs | rdis-vmserver | example volume to be mounted as datastore and replicated to DR array | +| rdis-vmshelf03 | DR\_DE4000H\_vmcluster_admin | DR\_DE4000H\_vmfs | rdis-vmserver | admin volume mounted as datastore on Primary and DR clusters, for a planned DR / failover the essential virtual machines can be moved via vMotion here. | +| rdis-vmshelf03 | PRI\_DE2000H\_vol_repl | DR\_DE4000H\_vmfs | rdis-vmserver | In full DR outage this will be promoted within the mirro from secondary to primary then mounted in vCentre on host rdis-vmserver04. Not mounted as a datastore at present. | +| rdis-vmshelf03 | PRI\_DE4000H\_vol_repl | DR\_DE4000H\_vmfs | rdis-vmserver | In full DR outage this will be promoted within the mirro from secondary to primary then mounted in vCentre on host rdis-vmserver04. Not mounted as a datastore at present. | + +> The replication volumes must be equal or larger in size to the source volume for a mirror. + +## Create Volumes on each array + +- Create new Volume, assign host later +- Create new workload or select existing workload (where previous created for another volume), workloads should be type 'VMware VMFS', the name is unimportant the respective names have been chosen to be easily identified, PRI\_DE2000H\_vmfs / PRI\_DE4000H\_vmfs / DR\_DE4000H\_vmfs +- Add new Volume, set size = 1000GiB, this can be expanded as required + +![50beabd20c5bf99279584a065680f624.png](_resources/582ffabd015f44eeb29293bcdd3d1df3.png) + +![234e0a07219da3558110d84204a8dc97.png](_resources/fec653b6ae7f49ed8931e195972f7711.png) + +## Assign Volumes to the new Host ACL + +Use the cluster object as the for the 'assigned to' host ACL. + +![f0665b7033045d9b9fe4322af491c1cc.png](_resources/8781d2121d9645f6bfcf6fd4caae503a.png) + +# Create VMware Datastores + +Create datastores on the on the new volumes presented over iSCSI. + +To help identify the Volume check the WWID of the volume in the storage array console. + +![cb116993a92e04a73c4bf68c7109c7fa.png](_resources/90228244dfeb462daa0891af0ab4f50e.png) + +Navigate to hosts and clusters and select any ESXi host (the new datastore will be present for all members of the cluster), select actions -> storage -> new datastore + +![5d2dbf3b8ceffde0980012b4bb76858b.png](_resources/cc1f21241bab402d83253e56f4664190.png) + +Identifying the storage device by the WWID set the name according to the above table. + +![1e4ae4212132173c9af9f187ad1f277f.png](_resources/48fa3f3bd2a244839e93fa718ee5ad79.png) + +Format as VMFS version 6, use all available partions and available space. + +Navigate to storage and ensure any local datastores are renamed to denote the host to which they are attached. + +The storage array datastores should be now listed. + +![3ddbac0cab030bb3d404ed350cbb3458.png](_resources/14aab6a6265541dca130d7311d014e32.png) + +## Migrate VCSA to shared storage + +The VCSA appliance is bound to the first ESXi host in the cluster residing upon a local datastore from the initial build.. + +Ensure the PRI\_DE2000H\_vmcluster_admin datastore has been created. + +Navigate to hosts and clusters, find the rdis-vcsa01 virtual machine, open context menu and select migrate. + +![444fb8549a24de38a106d19f7c9d3e7f.png](_resources/9b28829fa10d49848201ffff45ce5352.png) + +Select change both compute resource and storage. + +![3f54049ff3f1552aae91dd0dc5c05c1f.png](_resources/34f4c72b637c46818e8e0ddf677844ee.png) + +Select another host and the new admin datastore. The initial appliance was thin provisioned, no disk format change should be required but can be changed. Retain the destination port group. + +![f33b74e3147396d003ff1b00df7f4908.png](_resources/5747284b086c4c42bb899d27cb36279e.png) + +### Test HA failover of the VCSA appliance + +To verify the VCSA appliance is now decoupled from local storage set the ESXi host currently hosting the VCSA appliance into maintenance mode and migrate hosts to other ESXi cluster hosts. + +![8de13fedfe11a76887a0c6b0acc2f285.png](_resources/3d2b576c1e844a12b5ed8decaab9d9d8.png) + +## ![1d665a449c1e9ef89d32012fa1ef4f4e.png](_resources/b179b7bbe234465d90083994cb382c00.png) + +### Migrate a virtual machine to the DR cluster and DR storage + +Select the virtual machine context menu and migrate. + +![5876a1f658733f267e7136c65bb9d8a4.png](_resources/4c4cdc4174704dd3a78b1bf86dcd1a5d.png) + +Select both compute and storage resource change. + +![74e0acf5a1135380f1a20f87d86d2eb3.png](_resources/e86426870f9943e087956eaff35877e5.png) + +Select the DR host. + +![0d8229ffc1707ac2301b0e7fce290369.png](_resources/24037c5d0b264e408644431e24442c63.png) + +Select the DR storage. + +![72a85a381e6ba9cdf1349b2c4d5b9239.png](_resources/d77d7efdf8bb4041b0971f9d556f9643.png) + +The networks should not require changing as they have been extended to both clusters, the uplinks to customer core switches pass the storage VLAN. + +This operation is slower than a local cluster migration owing to the traversal across the core switches, although still acceptably fast. + +The vMotion task can (automatically) traverse only the vMotion network where the target + destination storage is not presented to both clusters if there is a requirement for this architecture. + +# Update cluster software + +| Web Console | SSH rdis-20 socks5 address | Internal FQDN | User | Pass | +| --- | --- | --- | --- | --- | +| Appliance administration | https://129.67.94.11:443 | https://rdis-vcsa01.robots.university.ac.uk | administrator@vcsa.robots.university.ac.uk | Password0 | +| Vsphere  administration | https://129.67.94.11:5480 | https://rdis-vcsa01.robots.university.ac.uk:5480 | root | Password0 | + +Ensure the rdis-20 ssh profile is started and your browser is using the local socks5 ssh proxy on 127.0.0.1:8080, edit the local hosts on your laptop to include the following entry: + +> 129.67.94.11 rdis-vcsa01.robots.university.ac.uk + +## vCenter update + +Login to the appliance administration console with the FQDN rather than the appliance IP and a use compatible browser (google Chrome seems to render the application better than Firefox). + +Navigate to update, select check updates -> check CDROM + URL. Relevant patches will be listed. + +![fd36e493ed7530d728837259f502f7f5.png](_resources/8ca55336cea04657a66914af97501bf6.png) + +Select the latest update then select run pre-update checks, this should pass validation, select stage and install, follow all prompts. + +![f75f21bbc7d781a98cef2bd2f1666861.png](_resources/525b0f8fcccf44cfb888cd072b93db3c.png) + +As this is a fresh install there is no backup of the vCenter server. + +The appliance will reboot, reconnect to the appliance administration console to follow the post install actions progress. + +## Virtual machine update + +Login to the vCenter administration console with the FQDN. + +Navigate to hosts and clusters -> Robots datacenter -> Primary cluster -> updates -> VM hardware. + +Select all virtual machines (currently VCSA appliance and HA state appliances), select upgrade to match host. + +![cecf47252a4bbdf21839fa151f9d4f0f.png](_resources/4f763cf38f2d4a1f9e5ae94178277d59.png) + +Virtual machines will be updated to compatibility version 11. + +The vcsa appliance will be unable to update itself to a later virtual machine type. + +### ![c7f1bc5f25777097c6d8a50fc25a5480.png](_resources/4f2fa8c3d7de4aca95005544c6bc6265.png) + +![a49394ff9705693e57f2fcbb382bb8c9.png](_resources/3087ee6291d5447888572d7a0343c46a.png) + +Select Esxi 7.0 profile for a version 17 virtual machine version. + +![615e2703a87ca05247d9ea5c9d3692e4.png](_resources/5e5edc3ad4764523848ae478cf8cd00f.png) + +Restart guest OS, vCentre will be unavailable until reboot has completed. Login directly to the ESXi host where the VCSA appliance is located to check progress if required. + +![d0826e3eb4ea1155d1ac068108760874.png](_resources/2b1f65ac7d914258922a6e9324a0f5f0.png) + +After reboot check the VCSA appliance compatability version in the vCentre console. + +![87fe5966cf25255db3c0ca3b8348511b.png](_resources/dc1b953d21b94bea91d831cbe7f8c905.png) + +## Esxi host update + +To assist in this update, HA should be enabled and the VCSA appliance should reside upon shared storage and tested to migrate successfully between ESXi hosts as they patch and reboot. + +Navigate to hosts and clusters, select the Robots datacenter, select updates -> baselines -> check compliance -> check pre-check remediation + +![678c65c212a968cab798b88840ca8a94.png](_resources/c2f867118c364cec89a50cfb4746f826.png) + +Apply patches for ESXi hosts, check all attached baseline profiles -> stage. + +![14ebc0530793b2c386898a5ccec0dbf8.png](_resources/2b9b501cc847447dbfe8872b670f7465.png) + +The stage task should complete without error, check remediate. ESXi hosts will enter maintenance mode and each host will begin update, this process is applied in series, expect ESXi hosts to become unavailable and virtual machines to migrate. + +![3164fc2b653ca53298d70f7debcf208b.png](_resources/c1f56a514a2b45b8a7dd1b28c8bc9328.png) + +Once complete all hosts should be compliant. + +![b0a0865fc6e5c59333be57044721e1d5.png](_resources/911274a1fc8f4b7994a61e67f50a6f7e.png) + +Note: With the DR single host cluster you may have a remediation warning that DRS/HA are disabled. + +# Build OCF Admin Virtual Machine + +Owing limited connectivity to the environment and the need to run some admin applications in local proximity to the cluster, a small server is required. + +Upload a guest operating system ISO to one of the ESXi hosts via ssh/scp, ideally to the DE4000H\_vmcluster\_admin datastore. + +## Create virtual machine + +Navigate to hosts and clusters, select the 'Primary' cluster, open the context menu and select new virtual machine. + +Name the machine 'OCF-admin' and place on the Robots datacenter and 'Primary' cluster. + +Select storage DE4000H\_vmcluster\_admin, select version 18 hardware compatibility, ESXi 7.0.1. + +Select the correct guest and guest OS version, example; Linux and Centos 7 (64-bit) / Linux and Ubuntu Linux (64-bit). + +Select compute requirements and ensure adequate adequate disk size is selected ~ 64GB should suffice for a desktop and admin applications. + +Configure the virtual hardware with appropriate compute resource and, select the DVP\_External\_Management_Network port group and mount the OS install ISO. + +![1a0e53383b435c128567c3c496a18bb7.png](_resources/9cf205014fe74671a94eba971775990c.png) + +Run through the guest installer, with the following attributes. + +| Attribute | Option | +| --- | --- | +| IP address | 129.67.94.25 | +| Netmask | 255.255.254.0 | +| Gateway | 129.67.95.254 | +| Name server address | 129.67.1.180 129.67.1.1 | +| Hostname | OCF-admin | +| Domain name | robots.university.ac.uk | +| User name | ocf | +| Password | Password0 | +| Enable OpenSSH server | yes | + +Edit virtual machine settings to remove/disconnect the boot ISO, reboot. + +SSH to the virtual machine ssh `ocf@129.67.94.25` ensure all upgrade packages are applied for kernel enhancements running on top of hypervisors. + +Edit virtual machine settings and add an additional network adapter to the DVP\_Management\_Network. + +![a0026ca22b51719eb1a3477ec3c578ac.png](_resources/980c94451b2743e3b4317dea2a81facc.png) + +SSH to the virtual machine `ssh ocf@129.67.94.25` and configure the secondary adapter for the isolated management network. + +Centos; edit /etc/sysconfig/network-scripts/ifcfg-ens192 and apply changed with `systemctl restart network` + +```sh +TYPE=Ethernet +BOOTPROTO=none +NAME=ens192 +DEVICE=ens192 +ONBOOT=yes +IPADDR=10.0.1.60 +PREFIX=24 +``` + +Ubuntu; edit /etc/netplan/01-netcfg.yaml and apply changes with `sudo netplan apply` + +```sh +network: + version: 2 + renderer: networkd + ethernets: + ens160: + addresses: [ 129.67.94.25/23 ] + gateway4: 129.67.95.254 + nameservers: + search: [ robots.university.ac.uk ] + addresses: + - "129.67.1.180" + - "129.67.1.1" + ens192: + addresses: [ 10.0.1.60/24 ] +``` + +Traceroute/ping and check ARP cache to ensure layer 2 connectivity to adjacent management interfaces of the VM servers and storage admin interfaces. + +## Install VMware tools (optional - not applied in this instance) + +This may overwrite open vmware-tools where auto configured by the operating system, many operating systems do not come out of the box with these packages and you may install open vmware-tools (Linux) or opt to install the proprietary VMware tools package as per this example (Windows/Linux). + +Propriety vmware-tools compile kernel modules, every kernel update requires a tools update. + +### Open vmware tools + +Centos; `yum install open-vm-tools` + +Ubuntu; `sudo apt-get install open-vm-tools` + +### Proprietary vmware tools + +Reboot the virtual machine, before the OS is loaded select to install VMware tools, this will mount an ISO containing the installer packages. + +![dc46e6be8812f032aee129dbf5cc8638.png](_resources/90631f0304a342c2819402633589b01c.png) + +Mount the ISO once the OS has started, extract the VMware tools installer, run the installer accepting all defaults and overwrite open vmware-tools packages. + +```sh +ocf@OCF-admin:~$ lsblk +NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT +sda 8:0 0 16G 0 disk +└─sda1 8:1 0 16G 0 part + ├─OCF--admin--vg-root 253:0 0 15G 0 lvm / + └─OCF--admin--vg-swap_1 253:1 0 980M 0 lvm [SWAP] +sr0 11:0 1 55.9M 0 rom +ocf@OCF-admin:~$ sudo mkdir /mnt/cdrom;sudo mount /dev/sr0 /mnt/cdrom +[sudo] password for ocf: +mount: /mnt/cdrom: WARNING: device write-protected, mounted read-only. +ocf@OCF-admin:~$ ll /mnt/cdrom/ +total 56855 +dr-xr-xr-x 2 root root 2048 Mar 25 2020 ./ +drwxr-xr-x 3 root root 4096 Jun 21 10:57 ../ +-r-xr-xr-x 1 root root 1976 Mar 25 2020 manifest.txt* +-r-xr-xr-x 1 root root 4943 Mar 25 2020 run_upgrader.sh* +-r--r--r-- 1 root root 56414224 Mar 25 2020 VMwareTools-10.3.22-15902021.tar.gz +-r-xr-xr-x 1 root root 872044 Mar 25 2020 vmware-tools-upgrader-32* +-r-xr-xr-x 1 root root 918184 Mar 25 2020 vmware-tools-upgrader-64* +ocf@OCF-admin:~$ sudo cp -a /mnt/cdrom/VMwareTools-10.3.22-15902021.tar.gz /tmp/ +ocf@OCF-admin:~$ cd /tmp +ocf@OCF-admin:/tmp$ tar -xvzf VMwareTools-10.3.22-15902021.tar.gz +ocf@OCF-admin:/tmp$ cd vmware-tools-distrib +ocf@OCF-admin:/tmp/vmware-tools-distrib$ sudo ./vmware-install.pl +``` + +## Install desktop + +See remote admin desktop. + +### Install Thinksystem SAN manager + +Obtain the following installer from the Lenovo website, this is a download for the DE storage series and will require a chassis serial number found in the array web admin console for download authorisation. + +Redhat/Centos based systems require `yum install -y redhat-lsb` + +> lnvgy\_utl\_sanmanager\_05.00.54.9013\_linux_x86-64.bin + +Run the installer from the Guacamole desktop for the graphical installer `sudo ./lnvgy_utl_sanmanager_05.00.54.9013_linux_x86-64.bin` + +![b196ff68cafd8de6f26f5ae69eb70a4d.png](_resources/972cf44e5ea74bfcb0e15f4eb11d827c.png) + +Follow the prompts accepting defaults unless listed below. + +- Disable autosupport, this will be enabled after configuration. +- Allow storage array connection without validating certificates, the arrays do not have CA validated certificates, they reside upon an isolated networks in way of compensating control. + +The application installs a systemctl unit file and listens on tcp 8080 / 8443. + +```sh +ocf@OCF-admin:~$ systemctl list-unit-files | grep -i think +ThinkSystemSANManager.service enabled +``` + +> https://129.67.94.25:8443/ + +Open a web browser on the Guacamole desktop and navigate to the application URL. The OCF-admin host does not have a full matching FQDN, the application does not readily accept external connections from a host that cannot be validated (this is likely configurable), launching from the Guacamole desktop allows the browsers to run on the same host as the application with both server and client validating hostname from the common OS. + +Set the password to `Password0` + +![592152599fdaebd62e93f88b2d6eb1b1.png](_resources/4cd3b4af5db64542b3867b3dff2cfccf.png) + +Discover all storage arrays on the management network. + +![6aaf54977af592f5ffd31399a12d346c.png](_resources/d442f6b634de46649c0b5732d1f7df6e.png) + +Select the discovered storage arrays, the certificates will be automatically trusted. + +![b27dfabd29fb4d0e91a42f67e0b49332.png](_resources/c074c43e79094b77b9c150ccaf2c7114.png) + +Enter storage array admin passwords. + +![9db7ddf14838b2345389e35bbccbb27a.png](_resources/ab1d60c3dd694a52bfd3836a35d7793e.png) + +Mirroring options are now available. + +![fb171b5ffed1271264b142db2f2c5c71.png](_resources/ca9da25935db4adf8efa210576b73086.png) + +### Install Thinksystem storage manager + +> There is no need to install this package in a production environment +> +> lnvgy\_utl\_storagemanager\_11.60.0A54.0007\_linux_x86-64.bin +> +> This will only install correctly on an rpm based system owing to rpm pre/post script actions +> +> This is only run to setup the new array with no configuration in conjunction with a local dhcp service, the utility will discover a new unconfigured array and push a minimal configuration intended to allow access to the the system manager web application running on the array controllers. +> +> [https://thinksystem.lenovofiles.com/storage/help/topic/thinksystem\_storage\_de\_himg\_11.60.2/ThinkSystem\_Storage\_DE_HIMG.pdf](https://thinksystem.lenovofiles.com/storage/help/topic/thinksystem_storage_de_himg_11.60.2/ThinkSystem_Storage_DE_HIMG.pdf) +> +> Thinksystem host utilities are a subset of this installer without the agent. + +Run the installer `chmod +x lnvgy_utl_storagemanager_11.60.0A54.0007_linux_x86-64.bin;sudo ./lnvgy_utl_storagemanager_11.60.0A54.0007_linux_x86-64.bin` +Disable the service once array is built. +systemctl stop smagent +systemctl disable smagent + +## Setup NTP daemon + +The storage replication mechanisms require synchronised time sources. + +TSMs require time sync with XClarity Administrator, the TSM will be populated with the same time source as XClarity, as they are on an isolated network they cannot use the University time servers so must target the OCF admin virtual machine which is multi-homed and times from the University ntp server (this VM is not timed from the hypervisor). + +Edit the chronyd config to include the RDIS ntp server and ensure the daemon listens on the isolated management network for the storage to sync against. + +```sh +# Use public servers from the pool.ntp.org project. +# Please consider joining the pool (http://www.pool.ntp.org/join.html). +server ntp0.robots.university.ac.uk +server ntp1.robots.university.ac.uk +server ntp2.robots.university.ac.uk +server ntp3.robots.university.ac.uk + +# Record the rate at which the system clock gains/losses time. +driftfile /var/lib/chrony/drift + +# Allow the system clock to be stepped in the first three updates +# if its offset is larger than 1 second. +makestep 1.0 3 + +# Enable kernel synchronization of the real-time clock (RTC). +rtcsync + +# Enable hardware timestamping on all interfaces that support it. +#hwtimestamp * + +# Increase the minimum number of selectable sources required to adjust +# the system clock. +#minsources 2 + +# Allow NTP client access from local network. +#allow 192.168.0.0/16 +allow 10.0.1.0/24 + +# Serve time even if not synchronized to a time source. +#local stratum 10 + +# Specify file containing keys for NTP authentication. +#keyfile /etc/chrony.keys + +# Specify directory for log files. +logdir /var/log/chrony + +# Select which information is logged. +#log measurements statistics tracking +``` + +Restart chronyd `systemctl enable chronyd;systemctl restart chronyd` + +## Setup mail relay + +The storage arrays are on an isolated network, in order to send alerts the OCF-admin host must run a simple relay. + +```sh +apt-get update && apt-get install postfix mailutils libsasl2-2 libsasl2-modules curl + +nano -cw /etc/postfix/main.cf + +inet_interfaces = 10.0.1.60, 127.0.0.1 +mynetworks = 127.0.0.0/8 10.0.1.0/24 +relayhost = mail.robots.university.ac.uk:25 +smtpd_relay_restrictions = + permit_mynetworks +smtpd_recipient_restrictions = + permit_mynetworks + reject_unauth_destination +smtpd_sender_restrictions = + check_sender_access hash:/etc/postfix/allowed_senders + warn_if_reject reject_unverified_sender + reject + +nano -cw /etc/postfix/allowed_senders + +root@robots.university.ac.uk permit +* reject + +postmap /etc/postfix/allowed_senders +systemctl enable postfix +systemctl restart postfix +``` + +# Create a content library + +## Upload OVA templates, browse for address for library import + +If you have local connectivity to the cluster, templates/ISOs maybe simply uploaded via the vCenter web console using 'local file' as the source, this step assumes you do not have local connectivity. + +Upload files to one of the ESXi hosts via ssh/scp, ideally to the DE4000H\_vmcluster\_admin datastore, enable SSH on the ESXi host as appropriate. + +Browse to the ESXi host at the 'folder' endpoint, you will be prompted for a local account credentials, root may only be available. + +The vCentre server must be able to route to the ESXi management interface, in this example the 129.67.94/23 external management address of the ESXi host is used. + +![5a0a8df55f38d82d0167b97f8d84dffc.png](_resources/12e8aa7447744e60b7c3c419eb00c008.png) + +Navigate to the datastore/folder containing the target file. + +![061d3f4d650ba33935f003afabe2836b.png](_resources/dc5de8894afb4db5a7d2c33919f302b8.png) + +Copy the link of the target file intended to be imported to content library, edit the link to include ESXi host credentials, the link will be used to import the file to the content library. + +| URL | URL modified with credential (special characters in the password entry should not be URL encoded) | +| --- | --- | +| [https://10.0.1.21/folder/ISO%2FXClarity\_administrator\_vmware\_appliance/lnvgy\_sw\_lxca\_83-3.3.0\_vmware\_x86-64.ova?dcPath=ha-datacenter&dsName=DE4000H\_vmcluster\_admin](https://10.0.1.21/folder/ISO%2fXClarity_administrator_vmware_appliance/lnvgy_sw_lxca_83-3.3.0_vmware_x86-64.ova?dcPath=ha-datacenter&dsName=DE4000H_vmcluster_admin) | [https://root:Password0@10.0.1.21/folder/ISO%2FXClarity\_administrator\_vmware\_appliance/lnvgy\_sw\_lxca\_83-3.3.0\_vmware\_x86-64.ova?dcPath=ha-datacenter&dsName=DE4000H\_vmcluster\_admin](https://root:Password0@10.0.1.21/folder/ISO%2fXClarity_administrator_vmware_appliance/lnvgy_sw_lxca_83-3.3.0_vmware_x86-64.ova?dcPath=ha-datacenter&dsName=DE4000H_vmcluster_admin) | + +## Add admin library + +Navigate to menu -> content libraries -> create. + +Name the content library 'Admin Library', create this on the vCentre server rdis-vcsa01.robots.ac.uk. + +Check local content library and accept all default. + +Select DE2000H\_vmcluster\_admin for storage. + +## Import OVA templates and ISOs + +Navigate to menu -> content libraries -> admin library -> actions -> import item + +![fcd4f93fad4111305b7ddcbea0605c6e.png](_resources/7a1e23e46d93468ea0b8f4b855e9f185.png) + +Select the templates/ISOs from URL. + +| Item name | Notes | Source File URL | +| --- | --- | --- | +| lnvgy\_sw\_lxca\_83-3.3.0\_vmware_x86-64.ova | XClarity Administrator | [https://root:Password0@129.67.94.20/folder/ISO%2FXClarity\_administrator\_vmware\_appliance/lnvgy\_sw\_lxca\_83-3.3.0\_vmware\_x86-64.ova?dcPath=ha-datacenter&dsName=DE4000H\_vmcluster\_admin](https://root:Password0@10.0.1.21/folder/ISO%2fXClarity_administrator_vmware_appliance/lnvgy_sw_lxca_83-3.3.0_vmware_x86-64.ova?dcPath=ha-datacenter&dsName=DE4000H_vmcluster_admin) | +| lnvgy\_sw\_vmuim\_520-7.2.0\_vmware_x86-64.ova | XClarity VMware Integrator | [https://root:Password0@129.67.94.20/folder/ISO%2FXClarity\_integrator\_vmware\_appliance/lnvgy\_sw\_vmuim\_520-7.2.0\_vmware\_x86-64.ova?dcPath=ha-datacenter&dsName=DE4000H\_vmcluster\_admin](https://root:Password0@10.0.1.21/folder/ISO%2fXClarity_integrator_vmware_appliance/lnvgy_sw_vmuim_520-7.2.0_vmware_x86-64.ova?dcPath=ha-datacenter&dsName=DE4000H_vmcluster_admin) | +| ubuntu-18.04.4-server-amd64.iso | Ubuntu 18.04 | https://releases.ubuntu.com/18.04/ubuntu-18.04.5-live-server-amd64.iso | +| CentOS-7-x86_64-Minimal-2009.iso | Centos 7 | [http://mirror.vorboss.net/centos/7.9.2009/isos/x86\_64/CentOS-7-x86\_64-Minimal-2009.iso](http://mirror.vorboss.net/centos/7.9.2009/isos/x86_64/CentOS-7-x86_64-Minimal-2009.iso) | + +## ![c56cef84a73d6099295e4f81454e405b.png](_resources/3b9949f9e89a44c38ce7b6fd9d91a901.png) + +![e4d4b3dc03e80ab3f1b7427dfc5513db.png](_resources/e3c4d68eabdb4049b8b105a80ab052f4.png) + +Now that the content library is populated you may delete the files from the datastore that you uploaded to the content library. Note the new content library folder on the datastore. + +## ![5ec1f6ec61641464c0e28f2c8a1a4b81.png](_resources/c0b7b94da55448568a45ea3729b315a8.png) + +# Configure Storage Alerts + +As the storage management interfaces are on an isolated network ensure the OCF-admin host is running a mail relay. + +For each storage array, navigate to settings -> alerts and add the mail server and recipient email address. + +Mail server: 10.0.1.60 + +Sender: root@robots.university.ac.uk + +Recipient: root@robots.university.ac.uk + +![d5aa15feb03d7afc5c99415530de6777.png](_resources/f9eb3290d3a646f89cc6885d0f73a713.png) + +# Configure vCenter Alerts + +From vCenter navigate to menu -> administration -> single sign on -> users and groups -> domain: vcsa.robots.university.ac.uk. + +Select the administrator user, check edit and populate the email field with: root@robots.university.ac.uk. + +![7e726b48552079eff98e2ae2d9f6954a.png](_resources/695c12ca7632429fb8ccdb38ce97953b.png) + +Navigate to menu -> hosts and cluster -> select the VCSA instance -> configure -> general -> mail -> edit. + +Enter the mail server: mail.robots.university.ac.uk and the mail sender: root@robots.university.ac.uk. + +![20915a7c8d64905af137849d7471d03e.png](_resources/9e65127f522d4844924839baa4a2fe19.png) + +# Configure ThinkSystem SAN Manager asynchronous replication + +Open the ThinkSystem SAN manager web application, the chromium bookmark is on the OCF jump host. + +| | | +| --- | --- | +| URL | https://129.67.94.25:8443 | +| User | admin | +| Password | Password0 | + +Discover hosts, trust certificates when asked. + +![3db881dcafe495356e6ed2aea22dcc92.png](_resources/837c805f91cb4c72ab7cf2ad04891889.png) + +![eb74bf464fa8143eb5a5ccdc9d0a5bd7.png](_resources/4f5698ead986466e982f7a2f550c291d.png) + +Enter the password for each array. + +![9db7ddf14838b2345389e35bbccbb27a.png](_resources/ab1d60c3dd694a52bfd3836a35d7793e.png) + +The inventory will now list the storage arrays. + +![7dfe7bbc80991168047418b09274e673.png](_resources/9c3002cab4c546ce8c39c236a470dffa.png) + +Select all storage arrays and check Launch. + +NOTE: Aspects of the mirror creation are performed on array system manager web consoles, when launched from SAN manager web tokens are passed to enable/authorise the action. + +![e9dd273f2b72ff51972a34407111220f.png](_resources/980534d1a0ec490ea9ffa07591519bff.png) + +## Create storage replication mirrors + +PUT DOCUMENTATION HERE - JOHN HAS PDF! - where can i find the link to this? + +- The following replication profiles are for illustration, the destination volumes must be at least the equivalent size as the source volume, in production it would be safest to ensure the destination volume is a fraction larger. +- There is no tuning involved in this example, default replication frequency if quite aggressive at 10 minutes, the default source volume reservation is 20%, the default destination volume reservation is 20%, these reservations exist directly on the pool thus taking space from the pool. +- With the DE storage series the target replication volume is in read only mode, for a DR cluster to use the volume it must be promoted to primary status and remounted manually, this procedure is intended for a full primary site outage and inherently requires failback planning. + +| Consistency group name | Source shelf | Source volume | Destination shelf | Destination volume | +| --- | --- | --- | --- | --- | +| PRI\_DE2000H\_vol_repl | rdis-vmshelf01 | PRI\_DE2000H\_vol | rdis-vmshelf03 | PRI\_DE2000H\_vol_repl | +| PRI\_DE4000H\_vol_repl | rdis-vmshelf01 | PRI\_DE4000H\_vol | rdis-vmshelf03 | PRI\_DE4000H\_vol_repl | +| PRI\_DE2000H\_vmcluster\_adm\_repl | rdis-vmshelf01 | PRI\_DE2000H\_vmcluster_admin | rdis_vmshelf02 | PRI\_DE2000H\_vmcluster\_adm\_repl | + +Create and asynchronous mirrored pair, synchronous mirrored pairs are only compatible with FC networks. + +![b3ef6a105b91b27bfec1c29ee036259f.png](_resources/cf0e014b287d49bca86089e1adbaa939.png) + +Check new mirror consistency group. + +![1fdbb72a9dccc3f36b2eec5224e801fd.png](_resources/f194e80d24bf48ceb8ae318ce5b6915f.png) + +Select a unique name for the consistency group and the remote target array to replicate to. + +![347f8dac287fde0920f884feb2fb2a62.png](_resources/d81612aca2d64df188fb9b147bfa5184.png) + +Select the source volume to replicate from. + +![0d780d478f0c53983a586d67717eef09.png](_resources/519bee72320747dfb6f60d0fedfde07c.png) + +Select the destination volume to replicate to. + +![620f33e4b2f79f443c7d0d8a86d02178.png](_resources/a47355b9b0e5438d8cbceae98ed98d61.png) + +Check operations in progress on the source array. Launch the system manager web console for the source array from SAN manager to see the progress. + +![408033cda2619702482bb8c465a80a69.png](_resources/04c7a1d8d0654a6095e0141ae3868f7b.png) + +Check operations in process on the destination array. Launch the system manager web console for the destination array from SAN manager to see the progress. + +![5a3d06f950090496c1604cf9cf76ee01.png](_resources/ec5afe57441d413a8a5ccec0a8c03d79.png) + +When replication between two primary switches we observe 10Gbps speeds, when replicating to the DR array we see 1GBps speeds. + +![6d21797442c27ae6340191f55cb5d3ea.png](_resources/0dac207ce1ed45cbaf010244aac62ba1.png) + +Replication occurs from controller to controller over a single channel (uplink), this is illustrated in the performance metrics. + +To make best use of bandwidth (for local switch/local site replication) ensure the source and target volumes are located on the same controllers on the arrays, multiple concurrent replication jobs between the same source and destination arrays should be balanced between array controllers. + +![071b79447697f03d3d3e4d1869ccaf91.png](_resources/0a4225c5837545bc85b6491804b0170e.png) + +Once volumes are synchronised and in an active replication schedule, you can see the amount of data to be replicated in the next replication event, this is a helpful metric when tuning the replication frequency and keeping your network administrators informed of the expected traffic profile. + +![cefdf07589fec1c3f3a34efcfd2cc60d.png](_resources/326e962aa9824b409b5be0c6700b4ee1.png) + +# XClarity VMware integration + +| Appliance | Template | Source | +| --- | --- | --- | +| XClarity Administrator | lnvgy\_sw\_lxca\_83-3.3.0\_vmware_x86-64.ova | https://datacentersupport.lenovo.com/us/en/solutions/lnvo-lxcaupd | +| XClarity VMware Integrator | lnvgy\_sw\_vmuim\_520-7.2.0\_vmware_x86-64.ova | https://datacentersupport.lenovo.com/gb/en/documents/lnvo-vmware | + +## Install XClarity administrator + +Navigate to menu -> content libraries -> admin library -> ovf & ova templates, select the XClarity Administrator template, select the context menu and check new VM from this template. + +![180e27dc39c125d84294468d4ed50b86.png](_resources/8a3fc84146eb46c88c81097390138ccc.png) + +Name the virtual machine 'XClarity-Administrator', select the Robots datacenter, select the 'Primary' cluster. + +Select deployment size small, select the DE4000H\_vmcluster\_admin datastore and select virtual disk format as thin. + +Select the DVP\_Management\_Network and set static IPv4 address allocation, the appliance does not have vSphere template integration with the IP address attributes and requires configuration from a DHCP service, there is no need to populate any static IP attributes in this step. + +![f43797e00b0ef9a0f85d48324d203259.png](_resources/fccc82ecea6f4134aa10b80199558320.png) + +### Create a temporary DHCP service on DVP\_Management\_Network + +SSH to the OCF-admin host, install some packages and build a tiny DHCP service. + +Firewalld or equivalent service should be stopped or a rule to allow DHCP present. `systemctl stop firewalld` + +> The docker daemon will lose the IPtables rules with this action, restart the docker service to remedy. + +```sh +ssh ocf@129.67.94.25 +su - +yum install dnsmasq +systemctl stop dnsmasq +systemctl disable dnsmasq +#start DHCP service +dnsmasq --no-daemon --port=0 --interface=ens192 --conf-file='' --dhcp-range=10.0.1.70,10.0.1.80,255.255.255.0,12h +``` + +Start the appliance from the vCenter console, watch the DHCP service output to find the appliance IP. + +![6e060e9e65ca2094710db77a4f5cc5c4.png](_resources/847111a4ba6e4ef9b72d397ac19ea683.png) + +Open the admin console and follow the setup wizard. + +> https://10.0.1.72/ + +- Accept the license. +- Create user account. + +> username: administrator +> password: Password0 + +- Return to initial setup (checkbox at bottom left of page). +- Configure network access, set IPv4 to static assignment and IPv6 to auto configuration. + +> IP address: 129.67.94.17 +> Network Mask: 255.255.254.0 +> Gateway: 129.67.95.254 + +- Save IP settings and restart networking. +- In the vCenter console move the primary network adapter to DVS\_External\_Management_Network. +- Open the Xclarity Administrator admin console and login with the new administrator account. + +> https://129.67.94.17 + +- The page will return to initial setup, check configure network access. +- Select the DNS tab, populate DNS attributes + +> Host name: XClarity +> Domain name: robots.university.ac.uk +> DNS operating mode: static +> DNS server: 129.67.1.180 +> DNS server: 129.67.1.1 + +- Save DNS settings. +- Return to initial setup. +- Configure date and time preferences. + +> UTC -00:00, Greenwich Mean Time Great Britain +> 24h clock format +> NTP servers ntp0.robots.university.ac.uk ntp1.robots.university.ac.uk ntp2.robots.university.ac.uk ntp3.robots.university.ac.uk + +- Return to initial setup. +- Service and support. + +> periodic upload data - no thanks +> Call home configuration - revist this with customer number in hand (skip step). +> Lenovo upload facility, set prefix to robots.university.ac.uk, set email to root@robots.university.ac.uk, apply and enable. +> Lenovo warrenty web service, enabled, apply. +> Service recovery password (if this management server cannot be recovered, use this password to collect logs and service data): Password0 + +- Return to initial setup. +- Start managing systems. + +> The SR635 and NE1072T devices will be auto discovered. + +### Licence the XClarity pro appliance + +> license file recieved by customer: lnvgy\_fod\_0038\_1212230389\_anyos_noarch.key + +[lnvgy\_fod\_0038\_1212230389\_anyos_noarch.key](_resources/475a9baca7df44b996e6ab9e7f950425.key) + +From the XClarity web console dashboard select administration -> licenses -> select files -> import and apply. + +### add secondary network adapter for BMC/storage communication + +Navigate to the XClarity dashboard, select administration -> network access -> ip settings. + +- Check two network devices detected, select Eth1 inteface. +- Add the following address information for Eth1. + +> IP address: 10.0.1.61 +> Network Mask: 255.255.255.0 + +- Change the NTP configuration to use the OCF-Admin virtual machine 10.0.1.60. +- Save IP settings. + +### Create entries for hardware devices + +| Site | Device | Model | Interface | Credentials | +| --- | --- | --- | --- | --- | +| Primary | Server | SR635 | TSM IPMI 10.0.1.11 | TSM user: USERID pass: Password0 | +| Primary | Server | SR635 | TSM IPMI 10.0.1.12 | TSM user: USERID pass: Password0 | +| Primary | Server | SR635 | TSM IPMI 10.0.1.13 | TSM user: USERID pass: Password0 | +| DR | Server | SR635 | TSM IPMI 10.0.1.41 | TSM user: USERID pass: Password0 | +| Primary | Switch | NE1072T | SSH 129.67.94.5 | user: admin pass: Password0 | +| DR | Switch | NE1072T | SSH 129.67.94.7 | user: admin pass: Password0 | +| Primary | Storage | DE2000H controller A | HTTPS API 10.0.1.1 | user: admin pass: Password0 | +| Primary | Storage | DE2000H controller B | HTTPS API 10.0.1.2 | user: admin pass: Password0 | +| Primary | Storage | DE4000H controller A | HTTPS API 10.0.1.3 | user: admin pass: Password0 | +| Primary | Storage | DE4000H controller B | HTTPS API 10.0.1.4 | user: admin pass: Password0 | +| DR | Storage | DE4000H controller A | HTTPS API 10.0.1.31 | user: admin pass: Password0 | +| DR | Storage | DE4000H controller B | HTTPS API 10.0.1.32 | user: admin pass: Password0 | + +Naviagate to hardware -> discover and manage new devices -> manual input. + +Enter the IP of one of the devices in the table, check ok. + +Check use stored credentials, select manage stored credentials. + +Create a new stored credential for each of the users admin / USERID in the above table. + +![e941fe6d012e1121cb531f022ba1a6fd.png](_resources/5b55e422a591422494797d9979924d77.png) + +Complete the entry by checking the manage button. + +Enter all Server IP addresses, select the appropriate credential. + +![3643c066cae341a9a4ed6699ac024d2e.png](_resources/dc78b5b55b684a2cb04eee39da59c078.png) + +![c8b360f9d170632e9a4d8b419bffba7e.png](_resources/b696c83cce8f422e9b3215d0b1632c60.png) + +Use the same procedure to add the remaining switches / storage hardware. + +Adding hardware devices can often be troublesome, generally regenerate the SSL certificates in the storage and TSM consoles and remove devices from the inventory with the force option, often restarting the appliance will clear devices that 'seem' to be unable to be removed/fail-removed from the inventory. + +### Configure Lenovo call home + +![c4817bbfafd8ae7386fb962ba161b524.png](_resources/335c1f944d834609b66627e4e6a139eb.png) + +### Configure customer alerts + +To be configured by the customer. + +![f6ac7d0efa9a5126fe72db9d0e6973b0.png](_resources/a7c9a05c8fd944278aaa3dacb49b4205.png) + +## Configure event forwarding + +Setup email alerts, navigate to monitoring -> event forwarding -> new item. + +Input the mail relay, port and recipient. + +![ab37d10fa7980b30b6d3943fa789f726.png](_resources/6147ad6b43374c92ac6a17ca4a432a43.png) + +Select all devices. + +![debb54ef45be8b663a2328f8c8e53bb7.png](_resources/295f358143584654a7464c27d2d6bcd8.png) + +Tune the level and class of alerts, there can be a lot of alert noise with informational classes of data. + +Warranty events have actually been disabled whilst server host licenses have not been applied in the license portal, this may not be a functional issue according to Lenovo pre-sales. + +![23828a727ada07a005c0bd05024a2c80.png](_resources/a8f3b2a8dc9145569a080a703b0cfb9f.png) + +Dont select a specific schedule to ensure alerts are always sent to the root@robots mailbox. + +![3ef409ca87a84ca9daafb44738c08482.png](_resources/9b3b23ffd4524804923526dff17f55f0.png) + +Once the profile is created the new profile will be shown and editable. + +![2f79a63539141f3cd0fcc491b2c3ed2f.png](_resources/3106eae6453d47c6ac62c795f1083bb1.png) + +## Install XClarity VMware integrator + +Navigate to menu -> content libraries -> admin library -> ovf & ova templates, select the XClarity Administrator template, select the context menu and check new VM from this template. + +![9de59c1fe1533c0b00f964f45ff92bb4.png](_resources/642165d2dc38450bb242427ef342ba29.png) + +Name the virtual machine 'XClarity-Integrator', select the Robots datacenter, select the 'Primary' cluster. + +Select the DE4000H\_vmcluster\_admin datastore and select virtual disk format as thin. + +Select the DVP\_Management\_Network and set dhcp/default IPv4 address allocation, the appliance does not have vSphere template integration with the IP address attributes and requires configuration from a DHCP service, there is no need to populate any static IP attributes in this step. + +### Create a temporary DHCP service on DVP\_Management\_Network + +SSH to the OCF-admin host, install some packages and build a tiny DHCP service. + +Firewalld or equivalent service should be stopped or a rule to allow DHCP present. `systemctl stop firewalld` + +> The docker daemon will lose the IPtables rules with this action, restart the docker service to remedy. + +```sh +ssh ocf@129.67.94.25 +su - +yum install dnsmasq +systemctl stop dnsmasq +systemctl disable dnsmasq +#start DHCP service +dnsmasq --no-daemon --port=0 --interface=ens192 --conf-file='' --dhcp-range=10.0.1.70,10.0.1.80,255.255.255.0,12h +``` + +Start the appliance from the vCenter console, watch the DHCP service output to find the appliance IP. + +![c4cc583b9763eb038013970907d87472.png](_resources/d2c9dccc20fc4e739a5fe684f62c6762.png) + +Open the admin console and follow the setup wizard. + +> [https://10.0.1.74/](https://10.0.1.72/) + +- Accept the license. +- Set host name and IP adress attributes + +> hostname: XClarity-lxci +> Domain name: do not configure a domain, without a FQDN resolvable via the dns servers PTR lookups in the com.lenovo.lxci-7.2.0.0 package install will fail +> DNS: 129.67.1.180,129.67.1.1 +> Eth0 IP: 129.67.94.18 +> Eth0 Netmask: 255.255.254.0 +> Default Gateway: 192.67.95.254 +> Eth1 IP: 10.0.1.62 +> +> Eth1 Netmask: 255.255.255.0 + +- In the vCenter console move the primary network adapter to DVS\_External\_Management_Network (leave the secondary adapter, this is not used). +- You may need to restart the guest OS in vCenter if networking does not automatically reconfigure. +- Open the Xclarity Integrator admin console to continue setup, the 10.0.1/24 range is used for BMC integration, the 129.67.94/23 range is used for vCenter integration. + +> Used for initial configuration +> +> [https://10.0.1.62](https://10.0.1.62 "https://10.0.1.62/") +> +> Used for subsequent configuration +> +> https://129.67.94.18 + +- Setup user account. + +> Username: Administrator +> Password: Password0 + +- Register the integrator appliance with vCenter, select vCenter connection -> register. + +> vCenter server: 129.67.94.11 (you may not use the FQDN here unless the integrator appliance IP is also has an FQDN) +> Username: administrator@vcsa.robots.university.ac.uk +> Password: Password0 + +- Configure date and time preferences. + +> Region: Europe +> Time Zone: London +> NTP server: ntp0.robots.university.ac.uk + +- Restart the guest OS in vCenter. +- Apply the latest patch. + +> https://support.lenovo.com/us/en/solutions/LNVO-VMWARE + +![ee239f272013540a25adc7519a43f91d.png](_resources/79dcadd59db6495e823520eef2559696.png) + +![01677aa3763da7705b67b02a7535739d.png](_resources/d897fc5946534a689ebe4c91c6dad65d.png) + +![37b5b9278b399920b6e67c7e632b3c61.png](_resources/37839c95f83242f2b216c1f37ab7ba3d.png) + +- Regenerate the server certificate, security settings -> server certificate. + +- Enter the primary IP of the appliance that will be used for vCenter registration as the common name and regenerate the server certificate. + ![1458b4731007a064efbd1b8a658a4e8e.png](_resources/44970045689c44ca9ca1939570c4a1f7.png) + +- Register the Integrator appliance with the vCenter VCSA server, use the Administrator@vcsa.robots.university.ac.uk root account. + ![ad8f1bf5d4e436fb31c9565e8f9ce7a2.png](_resources/fc28e367a1624ed58cb7ecdc20fb19e0.png) + +- Disable Lifecycle manager, this is not compatible with the SR635 server models. + + +![c1557ae37017387006700a890aecc670.png](_resources/cf7aead6369340fca4eda38ec2d2c82d.png) + +- Log in to the vCenter server over SSH, check the UI extensions are in place, if the file `.com.lenovo.lxci-7.2.0.0.incomplete` is present you likely have FQDN related issues in the integrator appliance networking. + +> SSH user: root +> SSH password: Password0 + +![63e236e2b648e8e195c457ae751e6c53.png](_resources/88baf5e9bf644c089cccc312171888b9.png) + +If there is no menu option for Xclarity Integrator restart the vCenter appliance to ensure the web service picks up the new extensions and reloads the additional web modules on boot. + +![651d3b1e0ae6e4c8dc79d21e9293e808.png](_resources/6d1652f7d80e4de6a6e30ae0a895e073.png) + +From the vCenter menu open Lenovo XClarity Integrator, select the XClarity integrator service and check edit. + +![67639d364fc758967c2a2cf560aea93e.png](_resources/ed31ccda0608432ba3a7c71141606afa.png) + +Populate the host username and password. + +> Host: 129.67.94.18 + +> User: Administrator + +> Password: Password0 + +![2c305c55d2b1827111fcccbb8c8ef869.png](_resources/ef90166c5f104a66ab74127ae6b25862.png) + +Select refresh, both the Xclarity Integrator and the vCenter servers should now have an online status. + +![ef83d24b1bbfd467cdf410b6e02411ba.png](_resources/701a552ac2b24c3d97ae87f6079ce0bb.png) + +On the XClarity Administrator appliance ensure the server certificate is regenerated with a common name matching the primary IP address of 129.67.94.17. + +![655250d2412b8dea1382ffcb03e09d63.png](_resources/83017c189e784994897e11da975f9242.png) + +Connect to the XClarity Integrator to the XClarity Administrator. + +This task is performed in vCenter, an embedded frame is opened to an administration console on the XClarity integrator in this step. + +![dd98d043bd94aa92d0b0bbe13fa52220.png](_resources/6e31341c98e54211a92a87abeed937ef.png) + +Enter the XClarity integrator IP address and root credentials, the registration process will ask to trust the XClarity Administration certificate, if the certificate is empty the XClarity Administrator certificate was not regenerated with its common name as the primary IP (or FQDN). + +![8f8ab36d7db2eb7d2e0adf81d3c9977d.png](_resources/65a17ace222747568403d3aaaed7651b.png) + +Close the registration console, the vCenter Lenovo XClarity Integrator pane will now show the status of all services. + +![4286406231e77f52fd0973ac8107c28d.png](_resources/ca1a0341f1cc4fb890b12e3684870228.png) + +The vCenter Xclarity integrator extension should now show 4 managed ESXi servers (there maybe a short wait whilst the integrator appliance relays the information to vCenter), where ESXi servers are not yet shown as managed servers, manually select them in the discover new servers pane, to manually add an ESXi host use the private management IP such as 10.0.1.11. + +![ec95bbf57cf09694cff6a4d256b9ce10.png](_resources/4124d6c59b68478e8db0b741f5bcac43.png) + +The vCenter XClarity extension will now show relevant information for the chassis. + +![3f3b8e348118eff74c4f5d9dd4059326.png](_resources/4bfa2f8956544c26a2742a8f0307bff8.png) + +# vCenter alert clearing + +## Host attestation alert + +The host attestation alarm requires the TPM mode set to 2.0 as per the installation instructions. This alarm can be addresses by enabling secure boot in the host bios, however this can only be done with a local monitor and keyboard not via the TPM KVM. We assume that the host is in a secured datacenter and disable the alarm from within vCenter. + +Navigate to hosts and clusters, select the VCSA domain, check configure -> alarm definitions -> host TPM attestation alarm, select disable. + +![76886c87236057a71db5be0a9761af1d.png](_resources/2051dda05d8e4e09999fa93deb0d0b3d.png) + +## Status of other host hardware objects + +Once vmware integration with the Lenovo agents are configured, you may see an cryptic error following multiple firmware updates and OS installs. + +![3ce89663c7e8c65bb87487a2287a43de.png](_resources/ec97fcf6bb284aeda0109a33dd7a0969.png) + +Navigate to hardware health to get more information. + +![53f7f07e634e55fd4d8510b7a760bf7a.png](_resources/e0f14b8e23d64b3abbb5e38f9a227be9.png) + +Log into each hosts TPM adapter to clear the event logs, it is likely full. + +Watch for non informational warnings such as failed hardware. + +![0881255e5e41b269dfc52bf96ba8e6ec.png](_resources/4ccfd1858ecd48da81b39de6d49e394b.png) + +# Storage array ASUP alerts + +The storage array management interfaces sit in isolated management network, snmp trap and API communication is facilitated between the storage array and the XClarity appliance which also has a listening interface in this network. + +Auto support is not enabled by default on the storage arrays whereby the primary vendor Lenovo/Netapp would receive storage array reports, currently critical alerts are picked up through XClarity and emails sent to the designated mailbox. + +Even with auto support disabled the auto support report is collected daily, this is useful for OCF engineers to collect and send to Lenovo in the event of hardware issues. + +The OCF jump box acts as an email relay for any external email alerts for the storage arrays. The ASUP alert messages are delivered over HTTPS by default, they can be reconfigured for SMTP, this should stop critical alert emails stating there is no path to Netapp. + +![8ecdfef3281742d9fde5369f9c979aba.png](_resources/b12b6357673143b1be8445d87db32015.png) + +> Set mail relay as 10.0.1.60 +> Set sender as root@robots.university.ac.uk, this is the only white listed sender for this relay + +Enable Auto support for the array. + +Configure the auto support delivery method. + +![a3a1b398d188f7038e9ca986caca6e33.png](_resources/f6b2c1bf4618469dbb4a2f77c532f73c.png) + +You may now disable auto support. + +This configuration can be also changed via the cli via the SMcli tool, enabling auto support and checking status seemed to work correctly however setting the delivery method via the cli failed - I suspect the command reference documentation does not match the syntax expected by the updated version of the tool. + +> [https://thinksystem.lenovofiles.com/storage/help/index.jsp?topic=%2Fthinksystem\_storage\_command\_line\_interface_11.60.3%2Foverview.html&lang=en/US](https://thinksystem.lenovofiles.com/storage/help/index.jsp?topic=%2Fthinksystem_storage_command_line_interface_11.60.3%2Foverview.html&lang=en/US) +> Navigate to the Auto support commands section + +Download from the SAN Manager web console and unzip. + +![b6e47b29ff2cd83a8eb1334ce5662191.png](_resources/1cb9d9d10c3544da9f221c3bcfbf0543.png) + +To check the auto support settings (useful when auto support is disabled in the web interface) run the following SMCli command. + +``` +[root@OCF-admin bin]# ./SMcli 10.0.1.1 -u admin@local -p Password0 -k -c "show storageArray autoSupport;" +Performing syntax check... + +Syntax check complete. + +Executing script... + +The AutoSupport feature is disabled on this storage array. +The AutoSupport OnDemand feature is disabled on this storage array. +The AutoSupport Remote Diagnostics feature is disabled on this storage array. + +Delivery Method: Email (SMTP) + Destination email address: autosupport@netapp.com + Mail relay server: 10.0.1.60 + Sender email address: root@robots.university.ac.uk + +The AutoSupport daily schedule preferred time range is from 12:00 AM to 11:59 PM. +The AutoSupport weekly schedule preferred time range is from 12:00 AM to 11:59 PM on any day of the week. + +AutoSupport Capable AutoSupport OnDemand Capable Chassis Serial Number Daily Schedule Weekly Schedule +Yes (disabled) Yes S4BXG548 09:18 AM 03:53 AM on Saturdays + +Script execution complete. + +SMcli completed successfully. +``` \ No newline at end of file diff --git a/Remote_admin_desktop.md b/Remote_admin_desktop.md new file mode 100755 index 0000000..98cf697 --- /dev/null +++ b/Remote_admin_desktop.md @@ -0,0 +1,502 @@ +Remote admin desktop + +# Remote admin desktop + +## Install desktop + +### Centos + +```sh +ocfuser@engs-28010:~$ ssh ocf@129.67.94.25 +su - +yum update +yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm +yum install yum install lxqt* openbox tigervnc-server firefox chromium --exclude=lxqt-*-devel +reboot +``` + +### Ubuntu + +```sh +ocfuser@engs-28010:~$ ssh ocf@129.67.94.25 +sudo apt-get -y update +sudo apt-get -y upgrade +sudo apt-get -y install lxqt qterminal openbox tigervnc-standalone-server firefox chromium-browser +#disable graphical boot +sudo systemctl set-default multi-user.target +#revert network control changes +sudo apt-get remove connman +sudo unlink /etc/resolv.conf +sudo systemctl enable systemd-networkd +sudo ln -s /run/systemd/resolve/resolv.conf /etc/resolv.conf +sudo systemd-resolve --status +sudo reboot +``` + +## Configure VNC daemon + +### Auto generate .Xauthority and populate user VNC config files + +Set password to `Password0` + +```sh +ocf@OCF-admin:~$ vncserver + +You will require a password to access your desktops. + +Password: +Verify: +Would you like to enter a view-only password (y/n)? n +/usr/bin/xauth: file /home/ocf/.Xauthority does not exist + +New 'OCF-admin.robots.university.ac.uk:1 (ocf)' desktop at :1 on machine OCF-admin.robots.university.ac.uk + +Starting applications specified in /etc/X11/Xvnc-session +Log file is /home/ocf/.vnc/OCF-admin.robots.university.ac.uk:1.log + +Use xtigervncviewer -SecurityTypes VncAuth -passwd /home/ocf/.vnc/passwd :1 to connect to the VNC server. +``` + +### Create VNC profile for desktop environment + +```sh +nano -cw ~/.vnc/xstartup + +#!/bin/sh +unset SESSION_MANAGER +#unset DBUS_SESSION_BUS_ADDRESS +exec openbox-session & +startlxqt & +[ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup +[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources +xsetroot -solid grey +vncconfig -iconic & +``` + +### Create systemd unit file for VNC on port 5901 + +```sh +sudo nano /etc/systemd/system/vncserver@.service + +[Unit] +Description=a wrapper to launch an X server for VNC +After=syslog.target network.target + +[Service] +Type=forking +User=ocf +Group=ocf +WorkingDirectory=/home/ocf + +ExecStartPre=-/usr/bin/vncserver -kill :%i > /dev/null 2>&1 +ExecStart=/usr/bin/vncserver -depth 24 -geometry 1920x1080 -interface 10.0.1.60 -localhost no :%i +#Centos vncserver is older, does not support -localhost switch +#ExecStart=/usr/bin/vncserver -depth 24 -geometry 1920x1080 -interface 10.0.1.60 :%i +ExecStop=/usr/bin/vncserver -kill :%i + +[Install] +WantedBy=multi-user.target + +sudo systemctl daemon-reload +``` + +### Start/Enable systemd unit file + +```sh +sudo systemctl start vncserver@1 +sudo systemctl enable vncserver@1 +``` + +## Install Docker + +### Centos + +```sh +su - +sudo yum remove docker \ + docker-client \ + docker-client-latest \ + docker-common \ + docker-latest \ + docker-latest-logrotate \ + docker-logrotate \ + docker-engine +sudo yum install -y yum-utils +sudo yum-config-manager \ + --add-repo \ + https://download.docker.com/linux/centos/docker-ce.repo +sudo yum install docker-ce docker-ce-cli containerd.io +sudo systemctl start docker +sudo systemctl enable docker +sudo usermod -G docker ocf +sudo docker run hello-world +``` + +### Ubuntu + +```sh +sudo apt-get remove docker docker-engine docker.io containerd runc +sudo apt-get install \ + apt-transport-https \ + ca-certificates \ + curl \ + gnupg-agent \ + software-properties-common +curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - +sudo apt-key fingerprint 0EBFCD88 +sudo add-apt-repository \ + "deb [arch=amd64] https://download.docker.com/linux/ubuntu \ + $(lsb_release -cs) \ + stable" +sudo apt-get update +sudo apt-get install docker-ce docker-ce-cli containerd.io +sudo systemctl start docker +sudo systemctl enable docker +sudo /etc/init.d/docker start +sudo usermod -G sudo,docker ocf +#logout/login for group membership to take effect on current user session +docker run hello-world +``` + +## Install docker-compose + +```sh +sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose +sudo chmod +x /usr/local/bin/docker-compose +``` + +## Prepare Guacamole database and self signed SSL cert + +### Create volume mount directories + +```sh +sudo mkdir -p /opt/guacamole-docker/init /opt/guacamole-docker/data /opt/guacamole-docker/drive /opt/guacamole-docker/record /opt/guacamole-docker/nginx/ssl +sudo chmod +x /opt/guacamole-docker/init +``` + +### Prep postgress data directory + +```sh +sudo su - +docker run --rm guacamole/guacamole /opt/guacamole/bin/initdb.sh --postgres > /opt/guacamole-docker/init/initdb.sql +exit +``` + +### Prep self signed cert + +```sh +sudo su - +openssl req -nodes -newkey rsa:2048 -new -x509 -keyout /opt/guacamole-docker/nginx/ssl/self-ssl.key -out /opt/guacamole-docker/nginx/ssl/self.cert -subj '/C=GB/ST=University/L=University/O=Eng/OU=Robots/CN=OCF-admin.robots.university.ac.uk/emailAddress=root@robots.university.ac.uk' +cat /opt/guacamole-docker/nginx/ssl/self-ssl.key >> /opt/guacamole-docker/nginx/ssl/robots.cert +cat /opt/guacamole-docker/nginx/ssl/self.cert >> /opt/guacamole-docker/nginx/ssl/robots.cert +rm -f /opt/guacamole-docker/nginx/ssl/self-ssl.key /opt/guacamole-docker/nginx/ssl/self.cert +exit +``` + +## Setup Guacamole as a Docker service + +> Useful docker-compose file template +> +> https://github.com/boschkundendienst/guacamole-docker-compose + +### Populate nginx daemon configuration file + +```sh +sudo nano -cw /opt/guacamole-docker/nginx/nginx.conf + +### AAA +user nginx; +worker_processes 1; + +error_log /var/log/nginx/error.log warn; +pid /var/run/nginx.pid; + + +events { + worker_connections 1024; +} + + +http { + include /etc/nginx/mime.types; + default_type application/octet-stream; + + log_format main '$remote_addr - $remote_user [$time_local] "$request" ' + '$status $body_bytes_sent "$http_referer" ' + '"$http_user_agent" "$http_x_forwarded_for"'; + + access_log /var/log/nginx/access.log main; + + sendfile on; + #tcp_nopush on; + + keepalive_timeout 65; + + #gzip on; + + include /etc/nginx/conf.d/*.conf; +} +``` + +### Populate nginx default config + +```sh +sudo nano -cw /opt/guacamole-docker/nginx/robots.conf + +### BBB +server { + listen 443 ssl http2; + server_name localhost; + + ssl_certificate /etc/nginx/ssl/self.cert; + ssl_certificate_key /etc/nginx/ssl/self.cert; + + ssl_protocols TLSv1.2 TLSv1.3; + ssl_prefer_server_ciphers on; + ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH"; + ssl_ecdh_curve secp384r1; + ssl_session_cache shared:SSL:10m; + ssl_session_tickets off; + ssl_stapling off; + ssl_stapling_verify off; +# resolver 8.8.8.8 8.8.4.4 valid=300s; +# resolver_timeout 5s; + + #charset koi8-r; + #access_log /var/log/nginx/host.access.log main; + + location / { + #forward to container + proxy_pass http://guacamole:8080/guacamole/; + proxy_buffering off; + proxy_http_version 1.1; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header Upgrade $http_upgrade; + proxy_set_header Connection $http_connection; + proxy_cookie_path /guacamole/ /; + access_log off; + # allow large uploads (default=1m) + # 4096m = 4GByte + client_max_body_size 4096m; +} + + #error_page 404 /404.html; + + # redirect server error pages to the static page /50x.html + # + error_page 500 502 503 504 /50x.html; + location = /50x.html { + root /usr/share/nginx/html; + } + +} +``` + +### Create docker compose file + +```sh +sudo mkdir -p /etc/docker/compose/guacamole +sudo nano -cw /etc/docker/compose/guacamole/docker-compose.yml + +# docker-compose file for Apache Guacamole +# created by PCFreak 2017-06-28 + +version: '2.0' + +# networks +# create a network 'guacnetwork_compose' in mode 'bridged' +networks: + guacnetwork_compose: + driver: bridge + +# services +services: + # guacd + guacd: + container_name: guacd_compose + image: guacamole/guacd + networks: + guacnetwork_compose: + restart: always + volumes: + - /opt/guacamole-docker/drive:/drive:rw + - /opt/guacamole-docker/record:/record:rw + # postgres + postgres: + container_name: postgres_guacamole_compose + environment: + PGDATA: /var/lib/postgresql/data/guacamole + POSTGRES_DB: guacamole_db + POSTGRES_PASSWORD: Password0 + POSTGRES_USER: guacamole_user + image: postgres + networks: + guacnetwork_compose: + restart: always + volumes: + - /opt/guacamole-docker/init:/docker-entrypoint-initdb.d:ro + - /opt/guacamole-docker/data:/var/lib/postgresql/data:rw + + # guacamole + guacamole: + container_name: guacamole_compose + depends_on: + - guacd + - postgres + environment: + GUACD_HOSTNAME: guacd + POSTGRES_DATABASE: guacamole_db + POSTGRES_HOSTNAME: postgres + POSTGRES_PASSWORD: Password0 + POSTGRES_USER: guacamole_user + image: guacamole/guacamole + links: + - guacd + networks: + guacnetwork_compose: + ports: +## enable next line if not using nginx +## - 8080:8080/tcp # Guacamole is on :8080/guacamole, not /. +## enable next line when using nginx + - 8080/tcp + restart: always + + # nginx + nginx: + container_name: nginx_guacamole_compose + restart: always + image: nginx + volumes: + - /opt/guacamole-docker/nginx/ssl/robots.cert:/etc/nginx/ssl/self.cert:ro + - /opt/guacamole-docker/nginx/nginx.conf:/etc/nginx/nginx.conf:ro + - /opt/guacamole-docker/nginx/robots.conf:/etc/nginx/conf.d/default.conf:ro + ports: + - 9443:443 + links: + - guacamole + networks: + guacnetwork_compose: + # run nginx + command: /bin/bash -c "nginx -g 'daemon off;'" +# nginx-debug-mode +# command: /bin/bash -c "nginx-debug -g 'daemon off;'" +``` + +### Create systemd unit files to start Guacamole Docker containers on boot + +```sh +sudo nano -cw /etc/systemd/system/docker-compose@.service + +[Unit] +Description=%i service with docker compose +Requires=docker.service +After=docker.service + +[Service] +Type=oneshot +RemainAfterExit=true +WorkingDirectory=/etc/docker/compose/%i +ExecStart=/usr/local/bin/docker-compose up -d --remove-orphans +ExecStop=/usr/local/bin/docker-compose down + +[Install] +WantedBy=multi-user.target + +sudo systemctl daemon-reload +``` + +### Start/Enable systemd unit file + +```sh +sudo systemctl start docker-compose@guacamole +sudo systemctl enable docker-compose@guacamole +``` + +## Setup NTP daemon + +Edit the chronyd config to include the RDIS ntp server and ensure the daemon listens on the isolated management network for the storage to use as a timesource. + +```sh +# Use public servers from the pool.ntp.org project. +# Please consider joining the pool (http://www.pool.ntp.org/join.html). +server ntp0.robots.university.ac.uk +server ntp1.robots.university.ac.uk +server ntp2.robots.university.ac.uk +server ntp3.robots.university.ac.uk + +# Record the rate at which the system clock gains/losses time. +driftfile /var/lib/chrony/drift + +# Allow the system clock to be stepped in the first three updates +# if its offset is larger than 1 second. +makestep 1.0 3 + +# Enable kernel synchronization of the real-time clock (RTC). +rtcsync + +# Enable hardware timestamping on all interfaces that support it. +#hwtimestamp * + +# Increase the minimum number of selectable sources required to adjust +# the system clock. +#minsources 2 + +# Allow NTP client access from local network. +#allow 192.168.0.0/16 +allow 10.0.1.0/24 + +# Serve time even if not synchronized to a time source. +#local stratum 10 + +# Specify file containing keys for NTP authentication. +#keyfile /etc/chrony.keys + +# Specify directory for log files. +logdir /var/log/chrony + +# Select which information is logged. +#log measurements statistics tracking +``` + +Restart chronyd `systemctl enable chronyd;systemctl restart chronyd` + +## Login to Guacamole, change credentials, create session + +### Edit Credentials + +| Attribute | Value | +| --- | --- | +| URL | [https://129.67.94.25:9443/](https://129.67.94.25:8443/) | +| Default User | guacadmin | +| Default Pass | guacadmin (changed to Password0) | +| OCF User | ocf | +| OCF Pass | Password0 | + +Edit settings. + +![6531057a0617b7b6dcb4a3c5f3ae5b58.png](_resources/32c98d85aa28442cba7d74af4a720c1d.png) + +Navigate to users -> new user. + +Create the OCF user, add all permissions to user, logout. + +Login as the new OCF user. + +Navigate to users -> guacadmin -> change password. + +### Create session + +Edit settings. + +Navigate to connections -> new connection + +| | | | +| --- | --- | --- | +| Name | OCF-admin | | +| Protocol | VNC | | +| Max connections | 1 | only one connection possible for a single VNC session (you may have additional view only sessions) | +| Max # connections per user | 1 | | +| Hostname | 10.0.1.60 | VNC started with -interface 10.0.1.60 -localhost no
Cannot listen on loopback as docker container will route to its own loopback not host, use isolated 10.0.1/24 range for better security. | +| Port | 5901 | systemd unit-file.1@service dynamic unit file starts at default port 5900 + dynamic unit name format 1 = 5901 | +| Username | ocf | set in ~/.vnc/passwd | +| Password | Password0 | set in ~/.vnc/passwd | \ No newline at end of file diff --git a/_resources/022d984329e64e8da11e2385fcae2fd8.png b/_resources/022d984329e64e8da11e2385fcae2fd8.png new file mode 100755 index 0000000..8d5eb75 Binary files /dev/null and b/_resources/022d984329e64e8da11e2385fcae2fd8.png differ diff --git a/_resources/03cf75cfb52a428ab0633a412fd23e0d.png b/_resources/03cf75cfb52a428ab0633a412fd23e0d.png new file mode 100755 index 0000000..09a8883 Binary files /dev/null and b/_resources/03cf75cfb52a428ab0633a412fd23e0d.png differ diff --git a/_resources/04c7a1d8d0654a6095e0141ae3868f7b.png b/_resources/04c7a1d8d0654a6095e0141ae3868f7b.png new file mode 100755 index 0000000..592516d Binary files /dev/null and b/_resources/04c7a1d8d0654a6095e0141ae3868f7b.png differ diff --git a/_resources/06dcc26e86ef4660b4f09b9ddb6e8f72.png b/_resources/06dcc26e86ef4660b4f09b9ddb6e8f72.png new file mode 100755 index 0000000..9aabf77 Binary files /dev/null and b/_resources/06dcc26e86ef4660b4f09b9ddb6e8f72.png differ diff --git a/_resources/0a4225c5837545bc85b6491804b0170e.png b/_resources/0a4225c5837545bc85b6491804b0170e.png new file mode 100755 index 0000000..a126209 Binary files /dev/null and b/_resources/0a4225c5837545bc85b6491804b0170e.png differ diff --git a/_resources/0c969f08ddfa4d2a9f9eb73b74f1db7f.png b/_resources/0c969f08ddfa4d2a9f9eb73b74f1db7f.png new file mode 100755 index 0000000..f057aca Binary files /dev/null and b/_resources/0c969f08ddfa4d2a9f9eb73b74f1db7f.png differ diff --git a/_resources/0da8a4d031bc44009a949b6979a90603.png b/_resources/0da8a4d031bc44009a949b6979a90603.png new file mode 100755 index 0000000..b2f34ee Binary files /dev/null and b/_resources/0da8a4d031bc44009a949b6979a90603.png differ diff --git a/_resources/0dac207ce1ed45cbaf010244aac62ba1.png b/_resources/0dac207ce1ed45cbaf010244aac62ba1.png new file mode 100755 index 0000000..eb8f3ff Binary files /dev/null and b/_resources/0dac207ce1ed45cbaf010244aac62ba1.png differ diff --git a/_resources/10b094b022664912b2bd775628b7f06f.png b/_resources/10b094b022664912b2bd775628b7f06f.png new file mode 100755 index 0000000..09884a7 Binary files /dev/null and b/_resources/10b094b022664912b2bd775628b7f06f.png differ diff --git a/_resources/11701df4a1454b23a672c7d7fb50990c.png b/_resources/11701df4a1454b23a672c7d7fb50990c.png new file mode 100755 index 0000000..1457585 Binary files /dev/null and b/_resources/11701df4a1454b23a672c7d7fb50990c.png differ diff --git a/_resources/12b5a9b56e9e4e299bccd8e137f2bc25.png b/_resources/12b5a9b56e9e4e299bccd8e137f2bc25.png new file mode 100755 index 0000000..fddaf55 Binary files /dev/null and b/_resources/12b5a9b56e9e4e299bccd8e137f2bc25.png differ diff --git a/_resources/12e8aa7447744e60b7c3c419eb00c008.png b/_resources/12e8aa7447744e60b7c3c419eb00c008.png new file mode 100755 index 0000000..d7505c7 Binary files /dev/null and b/_resources/12e8aa7447744e60b7c3c419eb00c008.png differ diff --git a/_resources/14aab6a6265541dca130d7311d014e32.png b/_resources/14aab6a6265541dca130d7311d014e32.png new file mode 100755 index 0000000..d65ab38 Binary files /dev/null and b/_resources/14aab6a6265541dca130d7311d014e32.png differ diff --git a/_resources/1644bc1070a4482795866192c3126190.png b/_resources/1644bc1070a4482795866192c3126190.png new file mode 100755 index 0000000..28da1c4 Binary files /dev/null and b/_resources/1644bc1070a4482795866192c3126190.png differ diff --git a/_resources/17d00a30a6364c40b583f05bd06b55d0.png b/_resources/17d00a30a6364c40b583f05bd06b55d0.png new file mode 100755 index 0000000..92712ea Binary files /dev/null and b/_resources/17d00a30a6364c40b583f05bd06b55d0.png differ diff --git a/_resources/1c68eaf4fef34f698abaf1e696199bb0.png b/_resources/1c68eaf4fef34f698abaf1e696199bb0.png new file mode 100755 index 0000000..42b8613 Binary files /dev/null and b/_resources/1c68eaf4fef34f698abaf1e696199bb0.png differ diff --git a/_resources/1cb9d9d10c3544da9f221c3bcfbf0543.png b/_resources/1cb9d9d10c3544da9f221c3bcfbf0543.png new file mode 100755 index 0000000..f08d557 Binary files /dev/null and b/_resources/1cb9d9d10c3544da9f221c3bcfbf0543.png differ diff --git a/_resources/1cc7db67b44e4b2081ab854878fea7a7.png b/_resources/1cc7db67b44e4b2081ab854878fea7a7.png new file mode 100755 index 0000000..e6cd4a0 Binary files /dev/null and b/_resources/1cc7db67b44e4b2081ab854878fea7a7.png differ diff --git a/_resources/2051dda05d8e4e09999fa93deb0d0b3d.png b/_resources/2051dda05d8e4e09999fa93deb0d0b3d.png new file mode 100755 index 0000000..e6846b0 Binary files /dev/null and b/_resources/2051dda05d8e4e09999fa93deb0d0b3d.png differ diff --git a/_resources/20ff58b793a94c81ba82ecf9d5928497.png b/_resources/20ff58b793a94c81ba82ecf9d5928497.png new file mode 100755 index 0000000..cf4808d Binary files /dev/null and b/_resources/20ff58b793a94c81ba82ecf9d5928497.png differ diff --git a/_resources/215fd990336b4659993ec39e09df65ce.png b/_resources/215fd990336b4659993ec39e09df65ce.png new file mode 100755 index 0000000..9192cc9 Binary files /dev/null and b/_resources/215fd990336b4659993ec39e09df65ce.png differ diff --git a/_resources/23d3e7d8b3da41e6a984a75f76603066.png b/_resources/23d3e7d8b3da41e6a984a75f76603066.png new file mode 100755 index 0000000..73bf884 Binary files /dev/null and b/_resources/23d3e7d8b3da41e6a984a75f76603066.png differ diff --git a/_resources/24037c5d0b264e408644431e24442c63.png b/_resources/24037c5d0b264e408644431e24442c63.png new file mode 100755 index 0000000..cb082fc Binary files /dev/null and b/_resources/24037c5d0b264e408644431e24442c63.png differ diff --git a/_resources/2418b30f6166401781c521301330c46b.png b/_resources/2418b30f6166401781c521301330c46b.png new file mode 100755 index 0000000..1e51138 Binary files /dev/null and b/_resources/2418b30f6166401781c521301330c46b.png differ diff --git a/_resources/255bfbb9bf2a4ddcb5dc2edae6f9483c.png b/_resources/255bfbb9bf2a4ddcb5dc2edae6f9483c.png new file mode 100755 index 0000000..3f35b64 Binary files /dev/null and b/_resources/255bfbb9bf2a4ddcb5dc2edae6f9483c.png differ diff --git a/_resources/2669a3466b30494a827b3b12fa222f12.png b/_resources/2669a3466b30494a827b3b12fa222f12.png new file mode 100755 index 0000000..ed31e8a Binary files /dev/null and b/_resources/2669a3466b30494a827b3b12fa222f12.png differ diff --git a/_resources/28e1992d26894a30a05ac41fa436608d.png b/_resources/28e1992d26894a30a05ac41fa436608d.png new file mode 100755 index 0000000..749de4c Binary files /dev/null and b/_resources/28e1992d26894a30a05ac41fa436608d.png differ diff --git a/_resources/295c9b73b80c4680814d416034174036.png b/_resources/295c9b73b80c4680814d416034174036.png new file mode 100755 index 0000000..1af90cc Binary files /dev/null and b/_resources/295c9b73b80c4680814d416034174036.png differ diff --git a/_resources/295f358143584654a7464c27d2d6bcd8.png b/_resources/295f358143584654a7464c27d2d6bcd8.png new file mode 100755 index 0000000..e3a6267 Binary files /dev/null and b/_resources/295f358143584654a7464c27d2d6bcd8.png differ diff --git a/_resources/2af89ded3c184ba793a8b6fd93c38648.png b/_resources/2af89ded3c184ba793a8b6fd93c38648.png new file mode 100755 index 0000000..ad6682b Binary files /dev/null and b/_resources/2af89ded3c184ba793a8b6fd93c38648.png differ diff --git a/_resources/2b1f65ac7d914258922a6e9324a0f5f0.png b/_resources/2b1f65ac7d914258922a6e9324a0f5f0.png new file mode 100755 index 0000000..8be969a Binary files /dev/null and b/_resources/2b1f65ac7d914258922a6e9324a0f5f0.png differ diff --git a/_resources/2b9b501cc847447dbfe8872b670f7465.png b/_resources/2b9b501cc847447dbfe8872b670f7465.png new file mode 100755 index 0000000..d09be27 Binary files /dev/null and b/_resources/2b9b501cc847447dbfe8872b670f7465.png differ diff --git a/_resources/2d505be2d62d4b2e98795d4103fa6239.png b/_resources/2d505be2d62d4b2e98795d4103fa6239.png new file mode 100755 index 0000000..c23a319 Binary files /dev/null and b/_resources/2d505be2d62d4b2e98795d4103fa6239.png differ diff --git a/_resources/3052e6be9e2c4824adccf3180fd629a7.png b/_resources/3052e6be9e2c4824adccf3180fd629a7.png new file mode 100755 index 0000000..b112650 Binary files /dev/null and b/_resources/3052e6be9e2c4824adccf3180fd629a7.png differ diff --git a/_resources/3087ee6291d5447888572d7a0343c46a.png b/_resources/3087ee6291d5447888572d7a0343c46a.png new file mode 100755 index 0000000..32a60a4 Binary files /dev/null and b/_resources/3087ee6291d5447888572d7a0343c46a.png differ diff --git a/_resources/30de07198d74431089f1212812d1e289.png b/_resources/30de07198d74431089f1212812d1e289.png new file mode 100755 index 0000000..cbdaf9f Binary files /dev/null and b/_resources/30de07198d74431089f1212812d1e289.png differ diff --git a/_resources/3106eae6453d47c6ac62c795f1083bb1.png b/_resources/3106eae6453d47c6ac62c795f1083bb1.png new file mode 100755 index 0000000..65a7fb0 Binary files /dev/null and b/_resources/3106eae6453d47c6ac62c795f1083bb1.png differ diff --git a/_resources/321a9036664e48948a1a36be18d3e03f.png b/_resources/321a9036664e48948a1a36be18d3e03f.png new file mode 100755 index 0000000..468df8c Binary files /dev/null and b/_resources/321a9036664e48948a1a36be18d3e03f.png differ diff --git a/_resources/326e962aa9824b409b5be0c6700b4ee1.png b/_resources/326e962aa9824b409b5be0c6700b4ee1.png new file mode 100755 index 0000000..72a2d44 Binary files /dev/null and b/_resources/326e962aa9824b409b5be0c6700b4ee1.png differ diff --git a/_resources/32c98d85aa28442cba7d74af4a720c1d.png b/_resources/32c98d85aa28442cba7d74af4a720c1d.png new file mode 100755 index 0000000..d93642e Binary files /dev/null and b/_resources/32c98d85aa28442cba7d74af4a720c1d.png differ diff --git a/_resources/335c1f944d834609b66627e4e6a139eb.png b/_resources/335c1f944d834609b66627e4e6a139eb.png new file mode 100755 index 0000000..134a031 Binary files /dev/null and b/_resources/335c1f944d834609b66627e4e6a139eb.png differ diff --git a/_resources/339d1d64f5aa437aa31a11ee78dcfdb1.png b/_resources/339d1d64f5aa437aa31a11ee78dcfdb1.png new file mode 100755 index 0000000..85134db Binary files /dev/null and b/_resources/339d1d64f5aa437aa31a11ee78dcfdb1.png differ diff --git a/_resources/34965da92ed24ab99aa1472f0ec9b970.png b/_resources/34965da92ed24ab99aa1472f0ec9b970.png new file mode 100755 index 0000000..56d1ce7 Binary files /dev/null and b/_resources/34965da92ed24ab99aa1472f0ec9b970.png differ diff --git a/_resources/34f4c72b637c46818e8e0ddf677844ee.png b/_resources/34f4c72b637c46818e8e0ddf677844ee.png new file mode 100755 index 0000000..819422f Binary files /dev/null and b/_resources/34f4c72b637c46818e8e0ddf677844ee.png differ diff --git a/_resources/35fa79a1019c41e791d621804eebb6fd.png b/_resources/35fa79a1019c41e791d621804eebb6fd.png new file mode 100755 index 0000000..9b03caf Binary files /dev/null and b/_resources/35fa79a1019c41e791d621804eebb6fd.png differ diff --git a/_resources/36e4bf32ab6545a38b9a93d25cc76665.png b/_resources/36e4bf32ab6545a38b9a93d25cc76665.png new file mode 100755 index 0000000..2d69596 Binary files /dev/null and b/_resources/36e4bf32ab6545a38b9a93d25cc76665.png differ diff --git a/_resources/37839c95f83242f2b216c1f37ab7ba3d.png b/_resources/37839c95f83242f2b216c1f37ab7ba3d.png new file mode 100755 index 0000000..5dccbfc Binary files /dev/null and b/_resources/37839c95f83242f2b216c1f37ab7ba3d.png differ diff --git a/_resources/38caa0982c2546baaeec64220fecefd5.png b/_resources/38caa0982c2546baaeec64220fecefd5.png new file mode 100755 index 0000000..a63c163 Binary files /dev/null and b/_resources/38caa0982c2546baaeec64220fecefd5.png differ diff --git a/_resources/3b9949f9e89a44c38ce7b6fd9d91a901.png b/_resources/3b9949f9e89a44c38ce7b6fd9d91a901.png new file mode 100755 index 0000000..f2af66d Binary files /dev/null and b/_resources/3b9949f9e89a44c38ce7b6fd9d91a901.png differ diff --git a/_resources/3bf8cc2363214be3941bc16883aba061.png b/_resources/3bf8cc2363214be3941bc16883aba061.png new file mode 100755 index 0000000..4a320ec Binary files /dev/null and b/_resources/3bf8cc2363214be3941bc16883aba061.png differ diff --git a/_resources/3d2b576c1e844a12b5ed8decaab9d9d8.png b/_resources/3d2b576c1e844a12b5ed8decaab9d9d8.png new file mode 100755 index 0000000..6cc3c88 Binary files /dev/null and b/_resources/3d2b576c1e844a12b5ed8decaab9d9d8.png differ diff --git a/_resources/4124d6c59b68478e8db0b741f5bcac43.png b/_resources/4124d6c59b68478e8db0b741f5bcac43.png new file mode 100755 index 0000000..f614baf Binary files /dev/null and b/_resources/4124d6c59b68478e8db0b741f5bcac43.png differ diff --git a/_resources/41c26ff2a41e4ea2a7331606a19b541c.png b/_resources/41c26ff2a41e4ea2a7331606a19b541c.png new file mode 100755 index 0000000..3e62c51 Binary files /dev/null and b/_resources/41c26ff2a41e4ea2a7331606a19b541c.png differ diff --git a/_resources/42eff0119677431c95cf8a1a9283ca05.png b/_resources/42eff0119677431c95cf8a1a9283ca05.png new file mode 100755 index 0000000..3d0a1e7 Binary files /dev/null and b/_resources/42eff0119677431c95cf8a1a9283ca05.png differ diff --git a/_resources/44970045689c44ca9ca1939570c4a1f7.png b/_resources/44970045689c44ca9ca1939570c4a1f7.png new file mode 100755 index 0000000..36f695d Binary files /dev/null and b/_resources/44970045689c44ca9ca1939570c4a1f7.png differ diff --git a/_resources/46a773bb5e0741b6973fa46af85fe38f.png b/_resources/46a773bb5e0741b6973fa46af85fe38f.png new file mode 100755 index 0000000..4662fb1 Binary files /dev/null and b/_resources/46a773bb5e0741b6973fa46af85fe38f.png differ diff --git a/_resources/478763c554fd4b4ab6ff18c3a79bfc09.png b/_resources/478763c554fd4b4ab6ff18c3a79bfc09.png new file mode 100755 index 0000000..229ecf3 Binary files /dev/null and b/_resources/478763c554fd4b4ab6ff18c3a79bfc09.png differ diff --git a/_resources/48fa3f3bd2a244839e93fa718ee5ad79.png b/_resources/48fa3f3bd2a244839e93fa718ee5ad79.png new file mode 100755 index 0000000..b3e13cb Binary files /dev/null and b/_resources/48fa3f3bd2a244839e93fa718ee5ad79.png differ diff --git a/_resources/4915eb00bb6d41958510ecbb0aeb6525.png b/_resources/4915eb00bb6d41958510ecbb0aeb6525.png new file mode 100755 index 0000000..c5af63a Binary files /dev/null and b/_resources/4915eb00bb6d41958510ecbb0aeb6525.png differ diff --git a/_resources/4bfa2f8956544c26a2742a8f0307bff8.png b/_resources/4bfa2f8956544c26a2742a8f0307bff8.png new file mode 100755 index 0000000..b3ccb4e Binary files /dev/null and b/_resources/4bfa2f8956544c26a2742a8f0307bff8.png differ diff --git a/_resources/4c4cdc4174704dd3a78b1bf86dcd1a5d.png b/_resources/4c4cdc4174704dd3a78b1bf86dcd1a5d.png new file mode 100755 index 0000000..86eb2fc Binary files /dev/null and b/_resources/4c4cdc4174704dd3a78b1bf86dcd1a5d.png differ diff --git a/_resources/4ccfd1858ecd48da81b39de6d49e394b.png b/_resources/4ccfd1858ecd48da81b39de6d49e394b.png new file mode 100755 index 0000000..d34dc54 Binary files /dev/null and b/_resources/4ccfd1858ecd48da81b39de6d49e394b.png differ diff --git a/_resources/4cd3b4af5db64542b3867b3dff2cfccf.png b/_resources/4cd3b4af5db64542b3867b3dff2cfccf.png new file mode 100755 index 0000000..f299202 Binary files /dev/null and b/_resources/4cd3b4af5db64542b3867b3dff2cfccf.png differ diff --git a/_resources/4dd2b7273b3b490aa2b99ad464ee844f.png b/_resources/4dd2b7273b3b490aa2b99ad464ee844f.png new file mode 100755 index 0000000..203b8b3 Binary files /dev/null and b/_resources/4dd2b7273b3b490aa2b99ad464ee844f.png differ diff --git a/_resources/4f2fa8c3d7de4aca95005544c6bc6265.png b/_resources/4f2fa8c3d7de4aca95005544c6bc6265.png new file mode 100755 index 0000000..5a88f5c Binary files /dev/null and b/_resources/4f2fa8c3d7de4aca95005544c6bc6265.png differ diff --git a/_resources/4f5698ead986466e982f7a2f550c291d.png b/_resources/4f5698ead986466e982f7a2f550c291d.png new file mode 100755 index 0000000..824d9a6 Binary files /dev/null and b/_resources/4f5698ead986466e982f7a2f550c291d.png differ diff --git a/_resources/4f763cf38f2d4a1f9e5ae94178277d59.png b/_resources/4f763cf38f2d4a1f9e5ae94178277d59.png new file mode 100755 index 0000000..d751be8 Binary files /dev/null and b/_resources/4f763cf38f2d4a1f9e5ae94178277d59.png differ diff --git a/_resources/518bb25d1df54c5f81d8ad1ef3996acd.png b/_resources/518bb25d1df54c5f81d8ad1ef3996acd.png new file mode 100755 index 0000000..ffcdbe6 Binary files /dev/null and b/_resources/518bb25d1df54c5f81d8ad1ef3996acd.png differ diff --git a/_resources/519bee72320747dfb6f60d0fedfde07c.png b/_resources/519bee72320747dfb6f60d0fedfde07c.png new file mode 100755 index 0000000..dd68307 Binary files /dev/null and b/_resources/519bee72320747dfb6f60d0fedfde07c.png differ diff --git a/_resources/51e469ed017140a2b3f3df2e84ba2396.png b/_resources/51e469ed017140a2b3f3df2e84ba2396.png new file mode 100755 index 0000000..2c6f22c Binary files /dev/null and b/_resources/51e469ed017140a2b3f3df2e84ba2396.png differ diff --git a/_resources/51e71a82980b45f19731a9abb6a0146f.png b/_resources/51e71a82980b45f19731a9abb6a0146f.png new file mode 100755 index 0000000..e1b8499 Binary files /dev/null and b/_resources/51e71a82980b45f19731a9abb6a0146f.png differ diff --git a/_resources/525b0f8fcccf44cfb888cd072b93db3c.png b/_resources/525b0f8fcccf44cfb888cd072b93db3c.png new file mode 100755 index 0000000..a989fa0 Binary files /dev/null and b/_resources/525b0f8fcccf44cfb888cd072b93db3c.png differ diff --git a/_resources/527a9ef2d1914702ba5bc13087a9d523.png b/_resources/527a9ef2d1914702ba5bc13087a9d523.png new file mode 100755 index 0000000..161709b Binary files /dev/null and b/_resources/527a9ef2d1914702ba5bc13087a9d523.png differ diff --git a/_resources/54c0314cb4374aac8196ea0b74d48813.png b/_resources/54c0314cb4374aac8196ea0b74d48813.png new file mode 100755 index 0000000..8dcc341 Binary files /dev/null and b/_resources/54c0314cb4374aac8196ea0b74d48813.png differ diff --git a/_resources/5605fa906855401e833663f2d05d669f.png b/_resources/5605fa906855401e833663f2d05d669f.png new file mode 100755 index 0000000..e648a05 Binary files /dev/null and b/_resources/5605fa906855401e833663f2d05d669f.png differ diff --git a/_resources/5747284b086c4c42bb899d27cb36279e.png b/_resources/5747284b086c4c42bb899d27cb36279e.png new file mode 100755 index 0000000..9ad2af3 Binary files /dev/null and b/_resources/5747284b086c4c42bb899d27cb36279e.png differ diff --git a/_resources/57b696f569ec465f872660e21c0e2ee7.png b/_resources/57b696f569ec465f872660e21c0e2ee7.png new file mode 100755 index 0000000..866f708 Binary files /dev/null and b/_resources/57b696f569ec465f872660e21c0e2ee7.png differ diff --git a/_resources/582ffabd015f44eeb29293bcdd3d1df3.png b/_resources/582ffabd015f44eeb29293bcdd3d1df3.png new file mode 100755 index 0000000..6da0a1d Binary files /dev/null and b/_resources/582ffabd015f44eeb29293bcdd3d1df3.png differ diff --git a/_resources/5a9e800f004542c1976f94e87250b550.png b/_resources/5a9e800f004542c1976f94e87250b550.png new file mode 100755 index 0000000..a0c1ba6 Binary files /dev/null and b/_resources/5a9e800f004542c1976f94e87250b550.png differ diff --git a/_resources/5b55e422a591422494797d9979924d77.png b/_resources/5b55e422a591422494797d9979924d77.png new file mode 100755 index 0000000..7a04eaa Binary files /dev/null and b/_resources/5b55e422a591422494797d9979924d77.png differ diff --git a/_resources/5e5edc3ad4764523848ae478cf8cd00f.png b/_resources/5e5edc3ad4764523848ae478cf8cd00f.png new file mode 100755 index 0000000..4cb6921 Binary files /dev/null and b/_resources/5e5edc3ad4764523848ae478cf8cd00f.png differ diff --git a/_resources/6054a228be5e4b68a94f7fa7a1da7ab6.png b/_resources/6054a228be5e4b68a94f7fa7a1da7ab6.png new file mode 100755 index 0000000..8789cc4 Binary files /dev/null and b/_resources/6054a228be5e4b68a94f7fa7a1da7ab6.png differ diff --git a/_resources/6147ad6b43374c92ac6a17ca4a432a43.png b/_resources/6147ad6b43374c92ac6a17ca4a432a43.png new file mode 100755 index 0000000..40e0ae2 Binary files /dev/null and b/_resources/6147ad6b43374c92ac6a17ca4a432a43.png differ diff --git a/_resources/6224bcfaac054e01b1987447cd9d3684.png b/_resources/6224bcfaac054e01b1987447cd9d3684.png new file mode 100755 index 0000000..f218e9c Binary files /dev/null and b/_resources/6224bcfaac054e01b1987447cd9d3684.png differ diff --git a/_resources/62cf7bde6921423b841f2ac6f64f76e6.png b/_resources/62cf7bde6921423b841f2ac6f64f76e6.png new file mode 100755 index 0000000..e158231 Binary files /dev/null and b/_resources/62cf7bde6921423b841f2ac6f64f76e6.png differ diff --git a/_resources/642165d2dc38450bb242427ef342ba29.png b/_resources/642165d2dc38450bb242427ef342ba29.png new file mode 100755 index 0000000..83b5b48 Binary files /dev/null and b/_resources/642165d2dc38450bb242427ef342ba29.png differ diff --git a/_resources/651bc4bac2ef4ad19d2647174dff9747.png b/_resources/651bc4bac2ef4ad19d2647174dff9747.png new file mode 100755 index 0000000..6c073ec Binary files /dev/null and b/_resources/651bc4bac2ef4ad19d2647174dff9747.png differ diff --git a/_resources/658d70f211ea4a3f8d10c35fd05fd032.png b/_resources/658d70f211ea4a3f8d10c35fd05fd032.png new file mode 100755 index 0000000..391a771 Binary files /dev/null and b/_resources/658d70f211ea4a3f8d10c35fd05fd032.png differ diff --git a/_resources/65a17ace222747568403d3aaaed7651b.png b/_resources/65a17ace222747568403d3aaaed7651b.png new file mode 100755 index 0000000..ece4aba Binary files /dev/null and b/_resources/65a17ace222747568403d3aaaed7651b.png differ diff --git a/_resources/676365faba3c4659b28cd12d11d10d42.png b/_resources/676365faba3c4659b28cd12d11d10d42.png new file mode 100755 index 0000000..7eae52b Binary files /dev/null and b/_resources/676365faba3c4659b28cd12d11d10d42.png differ diff --git a/_resources/695c12ca7632429fb8ccdb38ce97953b.png b/_resources/695c12ca7632429fb8ccdb38ce97953b.png new file mode 100755 index 0000000..2b50d9c Binary files /dev/null and b/_resources/695c12ca7632429fb8ccdb38ce97953b.png differ diff --git a/_resources/69724731f7f14657916850986dfdc4f9.png b/_resources/69724731f7f14657916850986dfdc4f9.png new file mode 100755 index 0000000..2fea318 Binary files /dev/null and b/_resources/69724731f7f14657916850986dfdc4f9.png differ diff --git a/_resources/6d1652f7d80e4de6a6e30ae0a895e073.png b/_resources/6d1652f7d80e4de6a6e30ae0a895e073.png new file mode 100755 index 0000000..e9d7023 Binary files /dev/null and b/_resources/6d1652f7d80e4de6a6e30ae0a895e073.png differ diff --git a/_resources/6e31341c98e54211a92a87abeed937ef.png b/_resources/6e31341c98e54211a92a87abeed937ef.png new file mode 100755 index 0000000..58ea717 Binary files /dev/null and b/_resources/6e31341c98e54211a92a87abeed937ef.png differ diff --git a/_resources/701a552ac2b24c3d97ae87f6079ce0bb.png b/_resources/701a552ac2b24c3d97ae87f6079ce0bb.png new file mode 100755 index 0000000..3ec6101 Binary files /dev/null and b/_resources/701a552ac2b24c3d97ae87f6079ce0bb.png differ diff --git a/_resources/7156faca53764780a2fe914c7668534a.png b/_resources/7156faca53764780a2fe914c7668534a.png new file mode 100755 index 0000000..c35cd8f Binary files /dev/null and b/_resources/7156faca53764780a2fe914c7668534a.png differ diff --git a/_resources/73013be72b3f458ebeea7d5bc5ab9b43.png b/_resources/73013be72b3f458ebeea7d5bc5ab9b43.png new file mode 100755 index 0000000..b2302ea Binary files /dev/null and b/_resources/73013be72b3f458ebeea7d5bc5ab9b43.png differ diff --git a/_resources/7481d5c09d27462ba7fc877d27839270.png b/_resources/7481d5c09d27462ba7fc877d27839270.png new file mode 100755 index 0000000..a5efa5d Binary files /dev/null and b/_resources/7481d5c09d27462ba7fc877d27839270.png differ diff --git a/_resources/79dcadd59db6495e823520eef2559696.png b/_resources/79dcadd59db6495e823520eef2559696.png new file mode 100755 index 0000000..8c71d66 Binary files /dev/null and b/_resources/79dcadd59db6495e823520eef2559696.png differ diff --git a/_resources/7a1e23e46d93468ea0b8f4b855e9f185.png b/_resources/7a1e23e46d93468ea0b8f4b855e9f185.png new file mode 100755 index 0000000..f1b29f1 Binary files /dev/null and b/_resources/7a1e23e46d93468ea0b8f4b855e9f185.png differ diff --git a/_resources/7ab1f45dee47408f965d336d5c790c13.png b/_resources/7ab1f45dee47408f965d336d5c790c13.png new file mode 100755 index 0000000..aecfd2f Binary files /dev/null and b/_resources/7ab1f45dee47408f965d336d5c790c13.png differ diff --git a/_resources/7d59afd025924e96b5f70109c4006e55.png b/_resources/7d59afd025924e96b5f70109c4006e55.png new file mode 100755 index 0000000..16d315c Binary files /dev/null and b/_resources/7d59afd025924e96b5f70109c4006e55.png differ diff --git a/_resources/7dd61b90021547a984850d481765c16e.png b/_resources/7dd61b90021547a984850d481765c16e.png new file mode 100755 index 0000000..5a05880 Binary files /dev/null and b/_resources/7dd61b90021547a984850d481765c16e.png differ diff --git a/_resources/80cdc5332e52444e849da2831e9e1584.png b/_resources/80cdc5332e52444e849da2831e9e1584.png new file mode 100755 index 0000000..a7fb869 Binary files /dev/null and b/_resources/80cdc5332e52444e849da2831e9e1584.png differ diff --git a/_resources/81e80982157f41a896d5ae0ba194c304.png b/_resources/81e80982157f41a896d5ae0ba194c304.png new file mode 100755 index 0000000..aa9bc04 Binary files /dev/null and b/_resources/81e80982157f41a896d5ae0ba194c304.png differ diff --git a/_resources/83017c189e784994897e11da975f9242.png b/_resources/83017c189e784994897e11da975f9242.png new file mode 100755 index 0000000..2309732 Binary files /dev/null and b/_resources/83017c189e784994897e11da975f9242.png differ diff --git a/_resources/837c805f91cb4c72ab7cf2ad04891889.png b/_resources/837c805f91cb4c72ab7cf2ad04891889.png new file mode 100755 index 0000000..e6aabe8 Binary files /dev/null and b/_resources/837c805f91cb4c72ab7cf2ad04891889.png differ diff --git a/_resources/847111a4ba6e4ef9b72d397ac19ea683.png b/_resources/847111a4ba6e4ef9b72d397ac19ea683.png new file mode 100755 index 0000000..fb6296b Binary files /dev/null and b/_resources/847111a4ba6e4ef9b72d397ac19ea683.png differ diff --git a/_resources/8781d2121d9645f6bfcf6fd4caae503a.png b/_resources/8781d2121d9645f6bfcf6fd4caae503a.png new file mode 100755 index 0000000..c90958f Binary files /dev/null and b/_resources/8781d2121d9645f6bfcf6fd4caae503a.png differ diff --git a/_resources/88baf5e9bf644c089cccc312171888b9.png b/_resources/88baf5e9bf644c089cccc312171888b9.png new file mode 100755 index 0000000..dcdb51f Binary files /dev/null and b/_resources/88baf5e9bf644c089cccc312171888b9.png differ diff --git a/_resources/8a3fc84146eb46c88c81097390138ccc.png b/_resources/8a3fc84146eb46c88c81097390138ccc.png new file mode 100755 index 0000000..906b995 Binary files /dev/null and b/_resources/8a3fc84146eb46c88c81097390138ccc.png differ diff --git a/_resources/8a6a3e4af5fb44208f53c8db64c1a79b.png b/_resources/8a6a3e4af5fb44208f53c8db64c1a79b.png new file mode 100755 index 0000000..a963052 Binary files /dev/null and b/_resources/8a6a3e4af5fb44208f53c8db64c1a79b.png differ diff --git a/_resources/8bacf6fbee4546dc848ef9554e089bff.png b/_resources/8bacf6fbee4546dc848ef9554e089bff.png new file mode 100755 index 0000000..085a01c Binary files /dev/null and b/_resources/8bacf6fbee4546dc848ef9554e089bff.png differ diff --git a/_resources/8bbd442bccb140468b1e3743567b4498.png b/_resources/8bbd442bccb140468b1e3743567b4498.png new file mode 100755 index 0000000..14ab368 Binary files /dev/null and b/_resources/8bbd442bccb140468b1e3743567b4498.png differ diff --git a/_resources/8c0878f7a0c74ce7bebd7f11952a7298.png b/_resources/8c0878f7a0c74ce7bebd7f11952a7298.png new file mode 100755 index 0000000..19936ae Binary files /dev/null and b/_resources/8c0878f7a0c74ce7bebd7f11952a7298.png differ diff --git a/_resources/8c22b2b544734b7a8c2f204a944887a2.png b/_resources/8c22b2b544734b7a8c2f204a944887a2.png new file mode 100755 index 0000000..6189f88 Binary files /dev/null and b/_resources/8c22b2b544734b7a8c2f204a944887a2.png differ diff --git a/_resources/8c8561c02bd044eaba9136fa966dab4e.png b/_resources/8c8561c02bd044eaba9136fa966dab4e.png new file mode 100755 index 0000000..c0fe8fe Binary files /dev/null and b/_resources/8c8561c02bd044eaba9136fa966dab4e.png differ diff --git a/_resources/8ca55336cea04657a66914af97501bf6.png b/_resources/8ca55336cea04657a66914af97501bf6.png new file mode 100755 index 0000000..e35a336 Binary files /dev/null and b/_resources/8ca55336cea04657a66914af97501bf6.png differ diff --git a/_resources/8ca7fea1dcd24fbda05aea32b3ccb8d9.png b/_resources/8ca7fea1dcd24fbda05aea32b3ccb8d9.png new file mode 100755 index 0000000..ce5cd9e Binary files /dev/null and b/_resources/8ca7fea1dcd24fbda05aea32b3ccb8d9.png differ diff --git a/_resources/90228244dfeb462daa0891af0ab4f50e.png b/_resources/90228244dfeb462daa0891af0ab4f50e.png new file mode 100755 index 0000000..34b02a0 Binary files /dev/null and b/_resources/90228244dfeb462daa0891af0ab4f50e.png differ diff --git a/_resources/90631f0304a342c2819402633589b01c.png b/_resources/90631f0304a342c2819402633589b01c.png new file mode 100755 index 0000000..2a66024 Binary files /dev/null and b/_resources/90631f0304a342c2819402633589b01c.png differ diff --git a/_resources/911274a1fc8f4b7994a61e67f50a6f7e.png b/_resources/911274a1fc8f4b7994a61e67f50a6f7e.png new file mode 100755 index 0000000..43e1a28 Binary files /dev/null and b/_resources/911274a1fc8f4b7994a61e67f50a6f7e.png differ diff --git a/_resources/930a6d6646814e279ce52b3350caf94d.png b/_resources/930a6d6646814e279ce52b3350caf94d.png new file mode 100755 index 0000000..5d8f2d9 Binary files /dev/null and b/_resources/930a6d6646814e279ce52b3350caf94d.png differ diff --git a/_resources/968b35a674c84fa89992b141ef598eb1.png b/_resources/968b35a674c84fa89992b141ef598eb1.png new file mode 100755 index 0000000..37e9373 Binary files /dev/null and b/_resources/968b35a674c84fa89992b141ef598eb1.png differ diff --git a/_resources/972cf44e5ea74bfcb0e15f4eb11d827c.png b/_resources/972cf44e5ea74bfcb0e15f4eb11d827c.png new file mode 100755 index 0000000..b89ff77 Binary files /dev/null and b/_resources/972cf44e5ea74bfcb0e15f4eb11d827c.png differ diff --git a/_resources/980534d1a0ec490ea9ffa07591519bff.png b/_resources/980534d1a0ec490ea9ffa07591519bff.png new file mode 100755 index 0000000..eec4103 Binary files /dev/null and b/_resources/980534d1a0ec490ea9ffa07591519bff.png differ diff --git a/_resources/980c94451b2743e3b4317dea2a81facc.png b/_resources/980c94451b2743e3b4317dea2a81facc.png new file mode 100755 index 0000000..9c45b32 Binary files /dev/null and b/_resources/980c94451b2743e3b4317dea2a81facc.png differ diff --git a/_resources/9ad4b039afd047898d6bb94a677a7696.png b/_resources/9ad4b039afd047898d6bb94a677a7696.png new file mode 100755 index 0000000..a436df4 Binary files /dev/null and b/_resources/9ad4b039afd047898d6bb94a677a7696.png differ diff --git a/_resources/9aefb3d2311a47588288689373257c2d.png b/_resources/9aefb3d2311a47588288689373257c2d.png new file mode 100755 index 0000000..1afa0bb Binary files /dev/null and b/_resources/9aefb3d2311a47588288689373257c2d.png differ diff --git a/_resources/9b28829fa10d49848201ffff45ce5352.png b/_resources/9b28829fa10d49848201ffff45ce5352.png new file mode 100755 index 0000000..7c494ac Binary files /dev/null and b/_resources/9b28829fa10d49848201ffff45ce5352.png differ diff --git a/_resources/9b3b23ffd4524804923526dff17f55f0.png b/_resources/9b3b23ffd4524804923526dff17f55f0.png new file mode 100755 index 0000000..7462cdc Binary files /dev/null and b/_resources/9b3b23ffd4524804923526dff17f55f0.png differ diff --git a/_resources/9c3002cab4c546ce8c39c236a470dffa.png b/_resources/9c3002cab4c546ce8c39c236a470dffa.png new file mode 100755 index 0000000..9fde0cc Binary files /dev/null and b/_resources/9c3002cab4c546ce8c39c236a470dffa.png differ diff --git a/_resources/9cf205014fe74671a94eba971775990c.png b/_resources/9cf205014fe74671a94eba971775990c.png new file mode 100755 index 0000000..f873ee7 Binary files /dev/null and b/_resources/9cf205014fe74671a94eba971775990c.png differ diff --git a/_resources/9e65127f522d4844924839baa4a2fe19.png b/_resources/9e65127f522d4844924839baa4a2fe19.png new file mode 100755 index 0000000..39d7823 Binary files /dev/null and b/_resources/9e65127f522d4844924839baa4a2fe19.png differ diff --git a/_resources/a0546f88c8e44d62a1a7196fcc00ebf8.png b/_resources/a0546f88c8e44d62a1a7196fcc00ebf8.png new file mode 100755 index 0000000..37ef5bf Binary files /dev/null and b/_resources/a0546f88c8e44d62a1a7196fcc00ebf8.png differ diff --git a/_resources/a0b9cc8ba9b34a3495b5645edc931fca.png b/_resources/a0b9cc8ba9b34a3495b5645edc931fca.png new file mode 100755 index 0000000..a64395b Binary files /dev/null and b/_resources/a0b9cc8ba9b34a3495b5645edc931fca.png differ diff --git a/_resources/a47355b9b0e5438d8cbceae98ed98d61.png b/_resources/a47355b9b0e5438d8cbceae98ed98d61.png new file mode 100755 index 0000000..a4d096f Binary files /dev/null and b/_resources/a47355b9b0e5438d8cbceae98ed98d61.png differ diff --git a/_resources/a590de9e6f3f4894bd9f7b945fefe415.png b/_resources/a590de9e6f3f4894bd9f7b945fefe415.png new file mode 100755 index 0000000..129ce6d Binary files /dev/null and b/_resources/a590de9e6f3f4894bd9f7b945fefe415.png differ diff --git a/_resources/a7c9a05c8fd944278aaa3dacb49b4205.png b/_resources/a7c9a05c8fd944278aaa3dacb49b4205.png new file mode 100755 index 0000000..2b8d7ac Binary files /dev/null and b/_resources/a7c9a05c8fd944278aaa3dacb49b4205.png differ diff --git a/_resources/a8f3b2a8dc9145569a080a703b0cfb9f.png b/_resources/a8f3b2a8dc9145569a080a703b0cfb9f.png new file mode 100755 index 0000000..98d0113 Binary files /dev/null and b/_resources/a8f3b2a8dc9145569a080a703b0cfb9f.png differ diff --git a/_resources/a90a5a623f2e43a8855329b4cb768c25.png b/_resources/a90a5a623f2e43a8855329b4cb768c25.png new file mode 100755 index 0000000..251eda3 Binary files /dev/null and b/_resources/a90a5a623f2e43a8855329b4cb768c25.png differ diff --git a/_resources/aa71851f6bb0447db8fe8aef71fe42f6.png b/_resources/aa71851f6bb0447db8fe8aef71fe42f6.png new file mode 100755 index 0000000..e36de85 Binary files /dev/null and b/_resources/aa71851f6bb0447db8fe8aef71fe42f6.png differ diff --git a/_resources/ab1d60c3dd694a52bfd3836a35d7793e.png b/_resources/ab1d60c3dd694a52bfd3836a35d7793e.png new file mode 100755 index 0000000..5d21010 Binary files /dev/null and b/_resources/ab1d60c3dd694a52bfd3836a35d7793e.png differ diff --git a/_resources/acf77b15bfe440aab3a023b9cf37ad5d.png b/_resources/acf77b15bfe440aab3a023b9cf37ad5d.png new file mode 100755 index 0000000..b96cfc7 Binary files /dev/null and b/_resources/acf77b15bfe440aab3a023b9cf37ad5d.png differ diff --git a/_resources/ae4ae87c717342d7af0ecd2852f17aeb.png b/_resources/ae4ae87c717342d7af0ecd2852f17aeb.png new file mode 100755 index 0000000..81c29d7 Binary files /dev/null and b/_resources/ae4ae87c717342d7af0ecd2852f17aeb.png differ diff --git a/_resources/af7371d134bb4ba490308c984bc96754.png b/_resources/af7371d134bb4ba490308c984bc96754.png new file mode 100755 index 0000000..a4137f9 Binary files /dev/null and b/_resources/af7371d134bb4ba490308c984bc96754.png differ diff --git a/_resources/b0943b689efe4084bca3621451f300e2.png b/_resources/b0943b689efe4084bca3621451f300e2.png new file mode 100755 index 0000000..81856c2 Binary files /dev/null and b/_resources/b0943b689efe4084bca3621451f300e2.png differ diff --git a/_resources/b12b6357673143b1be8445d87db32015.png b/_resources/b12b6357673143b1be8445d87db32015.png new file mode 100755 index 0000000..24eb18b Binary files /dev/null and b/_resources/b12b6357673143b1be8445d87db32015.png differ diff --git a/_resources/b179b7bbe234465d90083994cb382c00.png b/_resources/b179b7bbe234465d90083994cb382c00.png new file mode 100755 index 0000000..1883fb2 Binary files /dev/null and b/_resources/b179b7bbe234465d90083994cb382c00.png differ diff --git a/_resources/b1b7b264183946ba98a1906ed38172e4.png b/_resources/b1b7b264183946ba98a1906ed38172e4.png new file mode 100755 index 0000000..ab8b707 Binary files /dev/null and b/_resources/b1b7b264183946ba98a1906ed38172e4.png differ diff --git a/_resources/b38e5e7b436f454b9fc6c55f649658a0.png b/_resources/b38e5e7b436f454b9fc6c55f649658a0.png new file mode 100755 index 0000000..f3e478b Binary files /dev/null and b/_resources/b38e5e7b436f454b9fc6c55f649658a0.png differ diff --git a/_resources/b696c83cce8f422e9b3215d0b1632c60.png b/_resources/b696c83cce8f422e9b3215d0b1632c60.png new file mode 100755 index 0000000..c5283a9 Binary files /dev/null and b/_resources/b696c83cce8f422e9b3215d0b1632c60.png differ diff --git a/_resources/b7171ddc06bf4a2a978f9541bc0f130e.png b/_resources/b7171ddc06bf4a2a978f9541bc0f130e.png new file mode 100755 index 0000000..02782b0 Binary files /dev/null and b/_resources/b7171ddc06bf4a2a978f9541bc0f130e.png differ diff --git a/_resources/babba410c963400fa25f61e9744b16e5.png b/_resources/babba410c963400fa25f61e9744b16e5.png new file mode 100755 index 0000000..b23b70d Binary files /dev/null and b/_resources/babba410c963400fa25f61e9744b16e5.png differ diff --git a/_resources/bc8367c6731e4dd1b7acaf703361a4f9.png b/_resources/bc8367c6731e4dd1b7acaf703361a4f9.png new file mode 100755 index 0000000..17f60bb Binary files /dev/null and b/_resources/bc8367c6731e4dd1b7acaf703361a4f9.png differ diff --git a/_resources/bd5360cbb19b4082a1635ce94c69ba80.png b/_resources/bd5360cbb19b4082a1635ce94c69ba80.png new file mode 100755 index 0000000..028d105 Binary files /dev/null and b/_resources/bd5360cbb19b4082a1635ce94c69ba80.png differ diff --git a/_resources/be1faac136bb4960a932838a602e4619.png b/_resources/be1faac136bb4960a932838a602e4619.png new file mode 100755 index 0000000..12bb31b Binary files /dev/null and b/_resources/be1faac136bb4960a932838a602e4619.png differ diff --git a/_resources/bed23312a9984467b998634fe8ac8002.png b/_resources/bed23312a9984467b998634fe8ac8002.png new file mode 100755 index 0000000..0f1f0d1 Binary files /dev/null and b/_resources/bed23312a9984467b998634fe8ac8002.png differ diff --git a/_resources/c0387cff72cf45cfa3b822f96dfdbb18.png b/_resources/c0387cff72cf45cfa3b822f96dfdbb18.png new file mode 100755 index 0000000..9a59180 Binary files /dev/null and b/_resources/c0387cff72cf45cfa3b822f96dfdbb18.png differ diff --git a/_resources/c074c43e79094b77b9c150ccaf2c7114.png b/_resources/c074c43e79094b77b9c150ccaf2c7114.png new file mode 100755 index 0000000..e9f905e Binary files /dev/null and b/_resources/c074c43e79094b77b9c150ccaf2c7114.png differ diff --git a/_resources/c0b7b94da55448568a45ea3729b315a8.png b/_resources/c0b7b94da55448568a45ea3729b315a8.png new file mode 100755 index 0000000..b0b1dac Binary files /dev/null and b/_resources/c0b7b94da55448568a45ea3729b315a8.png differ diff --git a/_resources/c1d5eeb76e0140c4b8c4d401b4cff13c.png b/_resources/c1d5eeb76e0140c4b8c4d401b4cff13c.png new file mode 100755 index 0000000..a9524fc Binary files /dev/null and b/_resources/c1d5eeb76e0140c4b8c4d401b4cff13c.png differ diff --git a/_resources/c1f56a514a2b45b8a7dd1b28c8bc9328.png b/_resources/c1f56a514a2b45b8a7dd1b28c8bc9328.png new file mode 100755 index 0000000..784c8fc Binary files /dev/null and b/_resources/c1f56a514a2b45b8a7dd1b28c8bc9328.png differ diff --git a/_resources/c2f7ebc0ad2e4d11845fc6b5ec264189.png b/_resources/c2f7ebc0ad2e4d11845fc6b5ec264189.png new file mode 100755 index 0000000..c9abf17 Binary files /dev/null and b/_resources/c2f7ebc0ad2e4d11845fc6b5ec264189.png differ diff --git a/_resources/c2f867118c364cec89a50cfb4746f826.png b/_resources/c2f867118c364cec89a50cfb4746f826.png new file mode 100755 index 0000000..86c16e4 Binary files /dev/null and b/_resources/c2f867118c364cec89a50cfb4746f826.png differ diff --git a/_resources/c58e9161c2b14fdabc65ec2520df7e90.png b/_resources/c58e9161c2b14fdabc65ec2520df7e90.png new file mode 100755 index 0000000..b421f8c Binary files /dev/null and b/_resources/c58e9161c2b14fdabc65ec2520df7e90.png differ diff --git a/_resources/c62d133dbe7442a2878ce634876c8658.png b/_resources/c62d133dbe7442a2878ce634876c8658.png new file mode 100755 index 0000000..b383a1c Binary files /dev/null and b/_resources/c62d133dbe7442a2878ce634876c8658.png differ diff --git a/_resources/c70235ca5a094c249ad28b7c33c4d960.png b/_resources/c70235ca5a094c249ad28b7c33c4d960.png new file mode 100755 index 0000000..88cdd25 Binary files /dev/null and b/_resources/c70235ca5a094c249ad28b7c33c4d960.png differ diff --git a/_resources/c724a19cda234daeafa842ea89c59824.png b/_resources/c724a19cda234daeafa842ea89c59824.png new file mode 100755 index 0000000..feea558 Binary files /dev/null and b/_resources/c724a19cda234daeafa842ea89c59824.png differ diff --git a/_resources/ca1a0341f1cc4fb890b12e3684870228.png b/_resources/ca1a0341f1cc4fb890b12e3684870228.png new file mode 100755 index 0000000..34ca592 Binary files /dev/null and b/_resources/ca1a0341f1cc4fb890b12e3684870228.png differ diff --git a/_resources/ca9da25935db4adf8efa210576b73086.png b/_resources/ca9da25935db4adf8efa210576b73086.png new file mode 100755 index 0000000..93496ff Binary files /dev/null and b/_resources/ca9da25935db4adf8efa210576b73086.png differ diff --git a/_resources/cac0368153944bbf93f2b2211366ecdb.png b/_resources/cac0368153944bbf93f2b2211366ecdb.png new file mode 100755 index 0000000..146073a Binary files /dev/null and b/_resources/cac0368153944bbf93f2b2211366ecdb.png differ diff --git a/_resources/cc1f21241bab402d83253e56f4664190.png b/_resources/cc1f21241bab402d83253e56f4664190.png new file mode 100755 index 0000000..dd1aaf8 Binary files /dev/null and b/_resources/cc1f21241bab402d83253e56f4664190.png differ diff --git a/_resources/ce6599d4fa8843e58281772db506f7a2.png b/_resources/ce6599d4fa8843e58281772db506f7a2.png new file mode 100755 index 0000000..90ce56c Binary files /dev/null and b/_resources/ce6599d4fa8843e58281772db506f7a2.png differ diff --git a/_resources/cf0e014b287d49bca86089e1adbaa939.png b/_resources/cf0e014b287d49bca86089e1adbaa939.png new file mode 100755 index 0000000..f647cd0 Binary files /dev/null and b/_resources/cf0e014b287d49bca86089e1adbaa939.png differ diff --git a/_resources/cf7aead6369340fca4eda38ec2d2c82d.png b/_resources/cf7aead6369340fca4eda38ec2d2c82d.png new file mode 100755 index 0000000..e3b5aaa Binary files /dev/null and b/_resources/cf7aead6369340fca4eda38ec2d2c82d.png differ diff --git a/_resources/d185f915d58a4db08e91defe9665e79a.png b/_resources/d185f915d58a4db08e91defe9665e79a.png new file mode 100755 index 0000000..09b6586 Binary files /dev/null and b/_resources/d185f915d58a4db08e91defe9665e79a.png differ diff --git a/_resources/d2c9dccc20fc4e739a5fe684f62c6762.png b/_resources/d2c9dccc20fc4e739a5fe684f62c6762.png new file mode 100755 index 0000000..6428649 Binary files /dev/null and b/_resources/d2c9dccc20fc4e739a5fe684f62c6762.png differ diff --git a/_resources/d442f6b634de46649c0b5732d1f7df6e.png b/_resources/d442f6b634de46649c0b5732d1f7df6e.png new file mode 100755 index 0000000..b58e794 Binary files /dev/null and b/_resources/d442f6b634de46649c0b5732d1f7df6e.png differ diff --git a/_resources/d4445738410045ab86a6637f949f1a3a.png b/_resources/d4445738410045ab86a6637f949f1a3a.png new file mode 100755 index 0000000..4fed0aa Binary files /dev/null and b/_resources/d4445738410045ab86a6637f949f1a3a.png differ diff --git a/_resources/d77d7efdf8bb4041b0971f9d556f9643.png b/_resources/d77d7efdf8bb4041b0971f9d556f9643.png new file mode 100755 index 0000000..2069a96 Binary files /dev/null and b/_resources/d77d7efdf8bb4041b0971f9d556f9643.png differ diff --git a/_resources/d81612aca2d64df188fb9b147bfa5184.png b/_resources/d81612aca2d64df188fb9b147bfa5184.png new file mode 100755 index 0000000..ea0ab13 Binary files /dev/null and b/_resources/d81612aca2d64df188fb9b147bfa5184.png differ diff --git a/_resources/d8521d3e4c2d4d72bfef9dbb89e9ca12.png b/_resources/d8521d3e4c2d4d72bfef9dbb89e9ca12.png new file mode 100755 index 0000000..71f17ee Binary files /dev/null and b/_resources/d8521d3e4c2d4d72bfef9dbb89e9ca12.png differ diff --git a/_resources/d8873683b955480cbcdaa445b5f2e0e4.png b/_resources/d8873683b955480cbcdaa445b5f2e0e4.png new file mode 100755 index 0000000..bacce41 Binary files /dev/null and b/_resources/d8873683b955480cbcdaa445b5f2e0e4.png differ diff --git a/_resources/d897fc5946534a689ebe4c91c6dad65d.png b/_resources/d897fc5946534a689ebe4c91c6dad65d.png new file mode 100755 index 0000000..eefe871 Binary files /dev/null and b/_resources/d897fc5946534a689ebe4c91c6dad65d.png differ diff --git a/_resources/da77efd4b6904afeb143a98bcfd4ee25.png b/_resources/da77efd4b6904afeb143a98bcfd4ee25.png new file mode 100755 index 0000000..5d87321 Binary files /dev/null and b/_resources/da77efd4b6904afeb143a98bcfd4ee25.png differ diff --git a/_resources/dacc7e8a722041a1b1a0c620efa0ac48.png b/_resources/dacc7e8a722041a1b1a0c620efa0ac48.png new file mode 100755 index 0000000..02eb065 Binary files /dev/null and b/_resources/dacc7e8a722041a1b1a0c620efa0ac48.png differ diff --git a/_resources/dc1b953d21b94bea91d831cbe7f8c905.png b/_resources/dc1b953d21b94bea91d831cbe7f8c905.png new file mode 100755 index 0000000..12b9fc2 Binary files /dev/null and b/_resources/dc1b953d21b94bea91d831cbe7f8c905.png differ diff --git a/_resources/dc5de8894afb4db5a7d2c33919f302b8.png b/_resources/dc5de8894afb4db5a7d2c33919f302b8.png new file mode 100755 index 0000000..330d71a Binary files /dev/null and b/_resources/dc5de8894afb4db5a7d2c33919f302b8.png differ diff --git a/_resources/dc78b5b55b684a2cb04eee39da59c078.png b/_resources/dc78b5b55b684a2cb04eee39da59c078.png new file mode 100755 index 0000000..c233f59 Binary files /dev/null and b/_resources/dc78b5b55b684a2cb04eee39da59c078.png differ diff --git a/_resources/dd25e93ac646497b80518e34af7227d1.png b/_resources/dd25e93ac646497b80518e34af7227d1.png new file mode 100755 index 0000000..46bd2ef Binary files /dev/null and b/_resources/dd25e93ac646497b80518e34af7227d1.png differ diff --git a/_resources/ded1674d6c3e43cf98e4ac057c779181.png b/_resources/ded1674d6c3e43cf98e4ac057c779181.png new file mode 100755 index 0000000..9c87cbd Binary files /dev/null and b/_resources/ded1674d6c3e43cf98e4ac057c779181.png differ diff --git a/_resources/deec045da79e4f6ca7f4e55b71ddd808.png b/_resources/deec045da79e4f6ca7f4e55b71ddd808.png new file mode 100755 index 0000000..79fd749 Binary files /dev/null and b/_resources/deec045da79e4f6ca7f4e55b71ddd808.png differ diff --git a/_resources/e0f14b8e23d64b3abbb5e38f9a227be9.png b/_resources/e0f14b8e23d64b3abbb5e38f9a227be9.png new file mode 100755 index 0000000..a2cced3 Binary files /dev/null and b/_resources/e0f14b8e23d64b3abbb5e38f9a227be9.png differ diff --git a/_resources/e144fba8e1ab4a40a68d6c73e7094d51.png b/_resources/e144fba8e1ab4a40a68d6c73e7094d51.png new file mode 100755 index 0000000..1bdadf1 Binary files /dev/null and b/_resources/e144fba8e1ab4a40a68d6c73e7094d51.png differ diff --git a/_resources/e225399bab9747a2bfdee180d5cb8840.png b/_resources/e225399bab9747a2bfdee180d5cb8840.png new file mode 100755 index 0000000..eb411ff Binary files /dev/null and b/_resources/e225399bab9747a2bfdee180d5cb8840.png differ diff --git a/_resources/e270c7d1342b49a68827e9608fe61268.png b/_resources/e270c7d1342b49a68827e9608fe61268.png new file mode 100755 index 0000000..f4cf964 Binary files /dev/null and b/_resources/e270c7d1342b49a68827e9608fe61268.png differ diff --git a/_resources/e3c4d68eabdb4049b8b105a80ab052f4.png b/_resources/e3c4d68eabdb4049b8b105a80ab052f4.png new file mode 100755 index 0000000..89528be Binary files /dev/null and b/_resources/e3c4d68eabdb4049b8b105a80ab052f4.png differ diff --git a/_resources/e41205193a084698ad3fc28bed7931ff.png b/_resources/e41205193a084698ad3fc28bed7931ff.png new file mode 100755 index 0000000..e70709f Binary files /dev/null and b/_resources/e41205193a084698ad3fc28bed7931ff.png differ diff --git a/_resources/e60d37ed62594f3eabe7f8ff7efd5f1a.png b/_resources/e60d37ed62594f3eabe7f8ff7efd5f1a.png new file mode 100755 index 0000000..4a663dd Binary files /dev/null and b/_resources/e60d37ed62594f3eabe7f8ff7efd5f1a.png differ diff --git a/_resources/e86426870f9943e087956eaff35877e5.png b/_resources/e86426870f9943e087956eaff35877e5.png new file mode 100755 index 0000000..323049f Binary files /dev/null and b/_resources/e86426870f9943e087956eaff35877e5.png differ diff --git a/_resources/e9cf53169171448db8297a327bb56055.png b/_resources/e9cf53169171448db8297a327bb56055.png new file mode 100755 index 0000000..a5d6056 Binary files /dev/null and b/_resources/e9cf53169171448db8297a327bb56055.png differ diff --git a/_resources/ec5afe57441d413a8a5ccec0a8c03d79.png b/_resources/ec5afe57441d413a8a5ccec0a8c03d79.png new file mode 100755 index 0000000..da07b2b Binary files /dev/null and b/_resources/ec5afe57441d413a8a5ccec0a8c03d79.png differ diff --git a/_resources/ec97fcf6bb284aeda0109a33dd7a0969.png b/_resources/ec97fcf6bb284aeda0109a33dd7a0969.png new file mode 100755 index 0000000..b5f7418 Binary files /dev/null and b/_resources/ec97fcf6bb284aeda0109a33dd7a0969.png differ diff --git a/_resources/ed31ccda0608432ba3a7c71141606afa.png b/_resources/ed31ccda0608432ba3a7c71141606afa.png new file mode 100755 index 0000000..f92736c Binary files /dev/null and b/_resources/ed31ccda0608432ba3a7c71141606afa.png differ diff --git a/_resources/edd2a0f209f6448491c2de239110a049.png b/_resources/edd2a0f209f6448491c2de239110a049.png new file mode 100755 index 0000000..83a23b3 Binary files /dev/null and b/_resources/edd2a0f209f6448491c2de239110a049.png differ diff --git a/_resources/ef543104c7e24a5b873c68fd6a053a9f.png b/_resources/ef543104c7e24a5b873c68fd6a053a9f.png new file mode 100755 index 0000000..0d18a79 Binary files /dev/null and b/_resources/ef543104c7e24a5b873c68fd6a053a9f.png differ diff --git a/_resources/ef90166c5f104a66ab74127ae6b25862.png b/_resources/ef90166c5f104a66ab74127ae6b25862.png new file mode 100755 index 0000000..a283573 Binary files /dev/null and b/_resources/ef90166c5f104a66ab74127ae6b25862.png differ diff --git a/_resources/f194e80d24bf48ceb8ae318ce5b6915f.png b/_resources/f194e80d24bf48ceb8ae318ce5b6915f.png new file mode 100755 index 0000000..3237b27 Binary files /dev/null and b/_resources/f194e80d24bf48ceb8ae318ce5b6915f.png differ diff --git a/_resources/f2cc23913119460bb6508c189c288d06.png b/_resources/f2cc23913119460bb6508c189c288d06.png new file mode 100755 index 0000000..de918dd Binary files /dev/null and b/_resources/f2cc23913119460bb6508c189c288d06.png differ diff --git a/_resources/f35c81c82eb04b25904fa15aa4514387.png b/_resources/f35c81c82eb04b25904fa15aa4514387.png new file mode 100755 index 0000000..0a8bb24 Binary files /dev/null and b/_resources/f35c81c82eb04b25904fa15aa4514387.png differ diff --git a/_resources/f48ec38a008c42c78d775a2e5b90e163.png b/_resources/f48ec38a008c42c78d775a2e5b90e163.png new file mode 100755 index 0000000..8814fa0 Binary files /dev/null and b/_resources/f48ec38a008c42c78d775a2e5b90e163.png differ diff --git a/_resources/f62b897723f24f6483f92c145f266f56.png b/_resources/f62b897723f24f6483f92c145f266f56.png new file mode 100755 index 0000000..8a03180 Binary files /dev/null and b/_resources/f62b897723f24f6483f92c145f266f56.png differ diff --git a/_resources/f6b2c1bf4618469dbb4a2f77c532f73c.png b/_resources/f6b2c1bf4618469dbb4a2f77c532f73c.png new file mode 100755 index 0000000..a9cbdaf Binary files /dev/null and b/_resources/f6b2c1bf4618469dbb4a2f77c532f73c.png differ diff --git a/_resources/f7725b463d3f4799b4bca44e20fcecf2.png b/_resources/f7725b463d3f4799b4bca44e20fcecf2.png new file mode 100755 index 0000000..4d48117 Binary files /dev/null and b/_resources/f7725b463d3f4799b4bca44e20fcecf2.png differ diff --git a/_resources/f8ad1ce9677b420ba21268f53423dd4c.png b/_resources/f8ad1ce9677b420ba21268f53423dd4c.png new file mode 100755 index 0000000..bd3cc5d Binary files /dev/null and b/_resources/f8ad1ce9677b420ba21268f53423dd4c.png differ diff --git a/_resources/f9eb3290d3a646f89cc6885d0f73a713.png b/_resources/f9eb3290d3a646f89cc6885d0f73a713.png new file mode 100755 index 0000000..167b26b Binary files /dev/null and b/_resources/f9eb3290d3a646f89cc6885d0f73a713.png differ diff --git a/_resources/f9f741fa8af843639cfa429ed52f5695.png b/_resources/f9f741fa8af843639cfa429ed52f5695.png new file mode 100755 index 0000000..7a21788 Binary files /dev/null and b/_resources/f9f741fa8af843639cfa429ed52f5695.png differ diff --git a/_resources/fa03562970de47b1a1dc55296243293c.png b/_resources/fa03562970de47b1a1dc55296243293c.png new file mode 100755 index 0000000..498ea06 Binary files /dev/null and b/_resources/fa03562970de47b1a1dc55296243293c.png differ diff --git a/_resources/fa2fc8009ec9431da2c3d41097467540.png b/_resources/fa2fc8009ec9431da2c3d41097467540.png new file mode 100755 index 0000000..e263f27 Binary files /dev/null and b/_resources/fa2fc8009ec9431da2c3d41097467540.png differ diff --git a/_resources/fbc952d2e7ca4ad48eb3cbeed53a038b.png b/_resources/fbc952d2e7ca4ad48eb3cbeed53a038b.png new file mode 100755 index 0000000..fc8e0f3 Binary files /dev/null and b/_resources/fbc952d2e7ca4ad48eb3cbeed53a038b.png differ diff --git a/_resources/fc28e367a1624ed58cb7ecdc20fb19e0.png b/_resources/fc28e367a1624ed58cb7ecdc20fb19e0.png new file mode 100755 index 0000000..edbf750 Binary files /dev/null and b/_resources/fc28e367a1624ed58cb7ecdc20fb19e0.png differ diff --git a/_resources/fc8dbdc2707641c591e3dd8e89d48d4a.png b/_resources/fc8dbdc2707641c591e3dd8e89d48d4a.png new file mode 100755 index 0000000..c4fd929 Binary files /dev/null and b/_resources/fc8dbdc2707641c591e3dd8e89d48d4a.png differ diff --git a/_resources/fccc82ecea6f4134aa10b80199558320.png b/_resources/fccc82ecea6f4134aa10b80199558320.png new file mode 100755 index 0000000..a68931e Binary files /dev/null and b/_resources/fccc82ecea6f4134aa10b80199558320.png differ diff --git a/_resources/fd37e729ea804d939c4d56e099675433.png b/_resources/fd37e729ea804d939c4d56e099675433.png new file mode 100755 index 0000000..72a8055 Binary files /dev/null and b/_resources/fd37e729ea804d939c4d56e099675433.png differ diff --git a/_resources/fe41ea383f3749d08eee3202814774bc.png b/_resources/fe41ea383f3749d08eee3202814774bc.png new file mode 100755 index 0000000..17bc76b Binary files /dev/null and b/_resources/fe41ea383f3749d08eee3202814774bc.png differ diff --git a/_resources/fec653b6ae7f49ed8931e195972f7711.png b/_resources/fec653b6ae7f49ed8931e195972f7711.png new file mode 100755 index 0000000..61f4a16 Binary files /dev/null and b/_resources/fec653b6ae7f49ed8931e195972f7711.png differ diff --git a/self_signed_ssl_generation.md b/self_signed_ssl_generation.md new file mode 100755 index 0000000..f13feb3 --- /dev/null +++ b/self_signed_ssl_generation.md @@ -0,0 +1,130 @@ +# These certificates were not used in the build, + +## install the cloudflare SSL generation suite: + +``` +su - +mkdir -p certs/ca certs/xclarity-administrator certs/xclarity-integrator +cd certs + +VERSION=$(curl --silent "https://api.github.com/repos/cloudflare/cfssl/releases/latest" | grep '"tag_name"' | sed -E 's/.*"([^"]+)".*/\1/') +VNUMBER=${VERSION#"v"} +wget https://github.com/cloudflare/cfssl/releases/download/${VERSION}/cfssl_${VNUMBER}_linux_amd64 -O cfssl +chmod +x cfssl +sudo mv cfssl /usr/local/bin + +VERSION=$(curl --silent "https://api.github.com/repos/cloudflare/cfssl/releases/latest" | grep '"tag_name"' | sed -E 's/.*"([^"]+)".*/\1/') +VNUMBER=${VERSION#"v"} +wget https://github.com/cloudflare/cfssl/releases/download/${VERSION}/cfssljson_${VNUMBER}_linux_amd64 -O cfssljson +chmod +x cfssljson +sudo mv cfssljson /usr/local/bin +cfssljson -version +``` + +## create certificate authority CSR: + +``` +vi ca-csr.json + +{ + "CN": "RDIS Root CA", + "key": { + "algo": "rsa", + "size": 2048 + }, + "names": [ + { + "C": "GB", + "L": "University", + "O": "University", + "OU": "Robotics", + "ST": "England" + } + ] +} +``` + +## generate CA authority: + +``` +cfssl gencert -initca ca-csr.json | cfssljson -bare ./ca/ca - +``` + +## create cfssl profile to define certificate attribute profile "server" that will be used in client certificate generation: + +``` +vi cfssl-profile.json + +{ + "signing": { + "default": { + "expiry": "8760h" + }, + "profiles": { + "server": { + "usages": ["signing", "digital signing", "key encipherment", "server auth"], + "expiry": "8760h" + } + } + } +} +``` + +## create host CSR config for each appliance: + +``` +vi xclarity-administrator-csr.json + +{ + "CN": "xclarity-administrator", + "key": { + "algo": "rsa", + "size": 2048 + }, + "names": [ + { + "C": "GB", + "L": "University", + "O": "University", + "OU": "Robotics", + "ST": "England" + } + ] +} + +vi xclarity-integrator-csr.json + +{ + "CN": "xclarity-integrator", + "key": { + "algo": "rsa", + "size": 2048 + }, + "names": [ + { + "C": "GB", + "L": "University", + "O": "University", + "OU": "Robotics", + "ST": "England" + } + ] +} +``` + +## generate host certificates: + +``` +cfssl gencert -ca ./ca/ca.pem -ca-key ./ca/ca-key.pem -config cfssl-profile.json -profile=server -hostname=XClarity,129.67.94.17,10.0.1.61,127.0.0.1,localhost xclarity-administrator-csr.json | cfssljson -bare ./xclarity-administrator/xclarity-administrator + +cfssl gencert -ca ./ca/ca.pem -ca-key ./ca/ca-key.pem -config cfssl-profile.json -profile=server -hostname=XClarity-lxci,129.67.94.18,10.0.1.62,127.0.0.1,localhost xclarity-integrator-csr.json | cfssljson -bare ./xclarity-integrator/xclarity-integrator +``` + +## create full cert chain for each appliance: + +``` +cd /root/certs +cat ca/ca.pem >> xclarity-administrator/xclarity-administrator-chain.pem +cat xclarity-administrator/xclarity-administrator.pem >> xclarity-administrator/xclarity-administrator-chain.pem +cat xclarity-administrator/xclarity-administrator-key.pem >> xclarity-administrator/xclarity-administrator-chain.pem +``` \ No newline at end of file