tseed
/
bright_computing_ipower_pdu_integration_python
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
bright_computing_ipower_pdu.../README.md

282 lines
9.1 KiB
Markdown
Raw Permalink Normal View History

initial commit
2022-10-26 16:43:16 +00:00
# What is this?
Gather (non-native supported) PDU metrics via SNMP for Bright Computing, scrape SNMP to present metrics in compatible json format using a Python helper script and a custom SNMP MIB.
# Bright reference
https://support.brightcomputing.com/manuals/8.0/admin-manual.pdf
https://support.brightcomputing.com/manuals/8.1/developer-manual.pdf
# Dependencies
## install system dependencies to run easysnmp
NOTE: Loughborough Bright master node already includes packages "net-snmp-devel, gcc python-devel"
To install easysnmp with python pip some system dependencies are required, easysnmp uses net-snmp for enhanced performance and compatibility.
https://easysnmp.readthedocs.io/en/latest/
Install easysnp via pip in the python virtual env
## Install MIB for the ipower PDU used
```
sudo cp PDUSNMP_V1.05.00.mib /usr/share/snmp/mibs/PDUSNMP_V1.05.00.txt
sudo chmod 644 /usr/share/snmp/mibs/PDUSNMP_V1.05.00.txt
sudo sed -i 's/mibs :/# mibs :/g' /etc/snmp/snmp.conf # the Bright master servers did not need this step, your local host might
```
## test snmpwalk with the new MIB
```
snmpwalk -c public -r 5 -t 10 -v 1 192.168.10.161 PDUSNMP::pdu-Meters
```
### OID's to walk
```
snmpwalk -c public -r 5 -t 10 -v 1 192.168.10.161 PDUSNMP::ipt-pdu-SNMP #all fields returned, will likely hit a socket timeout before you reach the end of the walk
snmpwalk -c public -r 5 -t 10 -v 1 192.168.10.161 PDUSNMP::pdu-Configuration
snmpwalk -c public -r 5 -t 10 -v 1 192.168.10.161 PDUSNMP::pdu-Information
snmpwalk -c public -r 5 -t 10 -v 1 192.168.10.161 PDUSNMP::pdu-Configuration
snmpwalk -c public -r 5 -t 10 -v 1 192.168.10.161 PDUSNMP::slave-SiteTotals
snmpwalk -c public -r 5 -t 10 -v 1 192.168.10.161 PDUSNMP::slave-Server
snmpwalk -c public -r 5 -t 10 -v 1 192.168.10.161 PDUSNMP::pdu-Configuration
snmpwalk -c public -r 5 -t 10 -v 1 192.168.10.161 PDUSNMP::pdu-Information
snmpwalk -c public -r 5 -t 10 -v 1 192.168.10.161 PDUSNMP::pdu-Meters
snmpwalk -c public -r 5 -t 10 -v 1 192.168.10.161 PDUSNMP::pdu-sockets
snmpwalk -c public -r 5 -t 10 -v 1 192.168.10.161 PDUSNMP::pdu-Environmental
snmpwalk -c public -r 5 -t 10 -v 1 192.168.10.161 PDUSNMP::pdu-Security
snmpwalk -c public -r 5 -t 10 -v 1 192.168.10.161 PDUSNMP::pdu-Logs
snmpwalk -c public -r 5 -t 10 -v 1 192.168.10.161 PDUSNMP::slave-pdus
snmpwalk -c public -r 5 -t 10 -v 1 192.168.10.161 PDUSNMP::slave-Information
snmpwalk -c public -r 5 -t 10 -v 1 192.168.10.161 PDUSNMP::ipt-pdu-SNMP | grep -i uptime
snmpwalk -c public -r 5 -t 10 -v 1 192.168.10.161 PDUSNMP::ipt-pdu-SNMP | grep -i up
snmpwalk -c public -r 5 -t 10 -v 1 192.168.10.161 PDUSNMP::ipt-pdu-SNMP
snmpwalk -c public -r 5 -t 10 -v 1 192.168.10.161 PDUSNMP::slave-Status
snmpwalk -c public -r 5 -t 10 -v 1 192.168.10.161 PDUSNMP::ipt-pdu-SNMP | grep -i days
snmpwalk -c public -r 5 -t 10 -v 1 192.168.10.161 PDUSNMP::ipt-pdu-SNMP | grep -i d
snmpwalk -c public -r 5 -t 10 -v 1 192.168.10.161 PDUSNMP::pdu-sockets
snmpwalk -c public -r 5 -t 10 -v 1 192.168.10.161 PDUSNMP::ipt-pdu-SNMP | more
```
Key metrics are in milliamps / millvolts / watts, I have no idea how the PDU's calculate KWh
I believe there is a snmp field to set for the cost of a KWh unit, but i didnt see a resultant price field?
```
snmpwalk -c public -r 5 -t 10 -v 1 192.168.10.161 PDUSNMP::pdu-Meters
PDUSNMP::pdu-meter1-VRMS.0 = INTEGER: 2371 Volts RMS 237.1
PDUSNMP::pdu-meter1-IRMS.0 = INTEGER: 866 Amps RMS 8.66
PDUSNMP::pdu-meter1-KW.0 = INTEGER: 1964 KWs 1.964
```
# Local environment setup to work on the script
This environment is built to replicate the Loughborough Bright master node.
## setup local python environment to replicate bright master node
```
cd ~/WORK/OCF_GIT/python-snmp
mkdir src
cd src
wget https://www.python.org/ftp/python/2.7.5/Python-2.7.5.tgz
tar -xvzf Python-2.7.5.tgz
mkdir localpython
cd Python-2.7.5
./configure --prefix ~/WORK/OCF_GIT/python-snmp/src/localpython
make
make -n install # check youre not about to spam installer files to system directories
make install
```
## create virtual environment with local python and activate
```
cd ~/WORK/OCF_GIT/python-snmp
virtualenv venv -p ~/WORK/OCF_GIT/python-snmp/src/localpython/bin/python
```
## activate the local python virtual environment
```
cd ~/WORK/OCF_GIT/python-snmp
source venv/bin/activate
python --version
which pip
pip install easysnmp
deactivate # exit venv when you're done
```
# Run the script
There are two scripts:
* ipower-pdu.py this is the script used, it sets standalone Bright 'entities' to be monitored
* ipower-pdu-concept.py this was the first iteration of the script that did not work, the aim was to add metrics to existing pdu entities, Bright did not allow this
To run on the command line:
Edit the targets dictionary in the script, in the format `'bright entity' : 'target address/hostname'`.
There is an ipower pdu in the OCF office to test against `targets = {'ocf-pdu1': '192.168.10.161'}`.
```
# this will output the json document for bright to set monitoring fields per entity (entity, metric, value), used when the collection script is initialized by Bright
python ipower-pdy.py --initialize
# this will output the stats for each bright entity listed in the targets dict
python ipower-pdy.py
```
JSON output looks like:
```
[
{
"metric": "VRMS",
"value": 460.8,
"entity": "10.0.255.104"
},
{
"metric": "IPK",
"value": 5.15,
"entity": "10.0.255.104"
},
{
"metric": "KW",
"value": 0.721,
"entity": "10.0.255.104"
},
{
"metric": "IRMS",
"value": 3.24,
"entity": "10.0.255.104"
},
{
"metric": "VPK",
"value": 326.5,
"entity": "10.0.255.104"
}
]
```
# Setup Bright to use the script
As we are monitoring a pdu device that is already used/uptime-monitored by Bright we have to create some standalone 'entities' to monitor, crucially these must have different names than existing pdus, unfortunatly unlike server entities we cannot collect additional stats for 'powerdistributionunits' type entities thus this work around.
https://support.brightcomputing.com/manuals/8.1/developer-manual.pdf
* page 17 chapter 2.5 - collection data producers
* page 22 chapter 2.11 - Collection Monitoring Data Producers With Filter And Multiplexer
* page 22 chapter 2.12 - Collection Monitoring Data Producers For Standalone Entities
```
cp ipower-pdu.py /cm/local/apps/cmd/scripts/powerscripts/ipower-pdu.py
chmod 700 /cm/local/apps/cmd/scripts/powerscripts/ipower-pdu.py
```
## cmsh commands
* add monitoring script (ensure this is on both master01 + master02)
* set node execution filter (which node the script will run from, the active head node such as master001)
* check metrics are being collected
* change metric collection retension
* add a sample graph in bright view for each standalone entity (ipower-pdu1-4) for the KW metric
```
cmsh
# Add standalone targets
monitoring standalone
add ipower-pdu1
set type ipower-pdu
commit
exit
add ipower-pdu2
set type ipower-pdu
commit
exit
add ipower-pdu3
set type ipower-pdu
commit
exit
add ipower-pdu4
set type ipower-pdu
commit
exit
# Create the collection, assign the monitoring script
monitoring setup
add collection ipower-pdu
set script /cm/local/apps/cmd/scripts/powerscripts/ipower-pdu.py
set format JSON
set interval 1m
commit
# Create an execution filter to run the collection only on the active head node (master01)
nodeexecutionfilters
active
commit
exit
# Check monitoring is being collected
monitoring standalone
list
Name (key) Type
------------------------ ------------------------
ipower-pdu1 ipower-pdu
ipower-pdu2 ipower-pdu
ipower-pdu3 ipower-pdu
ipower-pdu4 ipower-pdu
use ipower-pdu1
latestmetricdata
Measurable Parameter Type Value Age State Info
------------ ------------ ------------ ---------- ---------- ---------- ----------
IPK ipower-pdu 25.65 A 7.86s filtered
IRMS ipower-pdu 17.05 A 7.86s filtered
KW ipower-pdu 3.794 KW 7.86s filtered
VPK ipower-pdu 648.4 V 7.86s filtered
VRMS ipower-pdu 901 V 7.86s filtered
# Set retension and use default consolidator policy
exit
measurable
list
set VPK consolidator default # we set the consolidator to consolidate RLE data (samples) to be consolidated with the bright default profile at frequencies of hours/days/weeks
set VPK maximalsamples 11520 # cannot set to 0 'infinite', this value amounts to a sample every minute for 8 days
set VPK maximalage 604800 # this value represents seconds in a week, all RLE data should be discarded after this time and only a subset available in the consolidator data
show VPK
set IPK consolidator default
set IPK maximalsamples 11520
set IPK maximalage 604800
show IPK
set KW consolidator default
set KW maximalsamples 11520
set KW maximalage 604800
show KW
set IRMS consolidator default
set IRMS maximalsamples 11520
set IRMS maximalage 604800
show IRMS
set VRMS consolidator default
set VRMS maximalsamples 11520
set VRMS maximalage 604800
show VRMS
commit
```