SNMP (Simple Network Management Protocol) is an internet protocol used to monitor and manage devices including servers, routers, switches and assorted other devices. It allows gathering a glut of data - this can be hardware information (eg. cpu temperature), network information (eg. interface speed) or software information (eg. number of HTTP requests).
However, to get at this information, it first needs to be addressable. SNMP itself does not define which bits of information are available. Instead it uses MIBs, or Management Information Bases which are basically hierarchies or trees of OIDs or object identifiers. SNMP is implemented in numerous devices including the devices we use for load balancing and shaping our traffic.
To load balance our main internet presence we use BIG-IP LTMs from F5. By default, it comes with a rather extensive MIB but sometimes doesn't have exactly what we want. It wasn't until version 11.2.0 that F5 introduced the ability to add custom OIDs to the MIB. Even better is that it lets us run and capture the output of shell commands on the device itself. This functionality gives exactly what is needed to get some data that otherwise wouldn't be automatically available. Without further ado, here is how to use this.
(Fair warning, doing this requires some knowledge of Tcl, but Tcl is a really easy language to pick up.)
First, some information about the OID structure
- the base OID for F5s BIG-IP devices is
.1.3.6.1.4.1.3375.2 - custom OIDs are added with the .100 suffix (ie.
.1.3.6.1.4.1.3375.2.100)
On startup, the SNMP daemon on the BIG-IP checks for a file called /config/snmp/custom_mib.tcl. This file contains the OID definitions and Tcl functions to be called when the OID is requested.
To add a new OID you first have to register it using the register_mib function
register_mib".<oid>"<tcl_function><type>
where oid is something like ".1", ".2", ".3.1", etc. tcl_function is the name of the function that you want to call. And finally the type of the OID being defined. There are four types supported: int, string, gauge, and counter.
Once we've registered a function, we need to then define that function. So further down in the file, section off a part of it for your custom functions.
For example:
proc time {}{set status [catch{exec date +%s} now]if{$status!=0}{return-1}else{return$now}}
It should be noted that the function will receive no arguments, so whatever processing needs to be done needs to be done without context.
It is recommended that any shell commands executed should be wrapped in a catch statement. This way the snmpd is protected slightly. Also, watch out for things like infinite loops or logic errors. Since the Tcl execution happens within the snmpd process, it is possible to do unhealthy things that can have an adverse impact on the daemon.
To pick up the changes to the custom_mib.tcl the SNMP daemon needs to be restarted (bigstart snmpd restart). And of course, the custom OID should be checked to make sure everything is working:
snmpwalk -Os -c public -v 2c localhost .1.3.6.1.4.1.3375.2.100.1
Remember to check the log file (/var/log/snmpd.log) for errors.
So while this functionality is interesting, it is much more interesting to see a practical application. The default F5 MIB does not include every bit of detail you might want - sometimes it is only retrievable via the interpreter/shell or even tmsh. So here is an example of harvesting the time in seconds since the last configuration update and making it available:
To get the time in seconds since the last configuration update, BASH can be used to call tmsh:
tmsh -c "list sys db configsync.localconfigtime one-line" | cut -d\" -f2 > /tmp/lastupdate
Unfortunately, this returns a string in which only a single field is needed. With a judicious use of cut the seconds can be extracted and stored. And since this information might be useful outside SNMP, it can be stored on the file system somewhere. To generate this file, putting this into the crontab works.
Now that the data is available in a file, all that is needed now is a simple Tcl function to return the data:
register_mib".1" config_age int proc config_age {}{set now_status [catch{exec date +%s} now]set config_status [catch{exec/bin/cat /tmp/lastupdate} config]if{$now_status==0 and $config_status==0}{set result [expr$now-$config]}else{set result -1}return$result}
Now this data is available via SNMP. Specifically it is available for Nagios to monitor and large discrepancies between last update time between BIG-IP devices can be alerted. This is just the tip of the iceberg. With some more Tcl knowledge, more complex information can be made available via SNMP. Of course, this was just a quick hack and using cron and temporary files might not be suitable to all use cases, but this does demonstrate the ease and hackability of extending the default MIB of BIG-IP devices.
To read further on customizing MIB entries for BIG-IP devices, take a look at the F5 knowledge base article.