Discovery Methods Implementation Guide

Printer-friendly versionPDF version

How to implment a new discovery method?

Take a deep breath and a big cup of coffee. To implement new discovery methods are needed good amount of skills in networking and also in xml/xslt transformations. The main idea behid iDiscover is to start discovery on a single network node. The algorithm is based on several discoveryMethods and each of them is run agaist the current node. Then it is run against the node neighbors and so on and so for.

In the guide is given an example with rfc 4292 (IP FORWARDING MIB). In order to find the device neighbors using IP Forwarding MIB have to be analyzed each route in device forwaring table. The neighbors are the next hops of each route. Since there might be many routes have to substracted the unique next hop values. Step by step process is split between iDiscover and iTopoManager and is presented below.

iDiscover

Step 1 - Determine the right request

Determine how to get the ip next hop raw data?  If we follow the rfc we will read that the best option for import is IpCidrRouteEntry. As per the RFC it contains the following SNMP OIDs.

IpCidrRouteEntry ::= SEQUENCE {
ipCidrRouteDestType InetAddressType,
ipCidrRouteDest InetAddress,
ipCidrRoutePfxLen InetAddressPrefixLength,
ipCidrRoutePolicy OBJECT IDENTIFIER,
ipCidrRouteNextHopType InetAddressType,
ipCidrRouteNextHop InetAddress,
ipCidrRouteIfIndex InterfaceIndexOrZero,
ipCidrRouteType INTEGER,
ipCidrRouteProto IANAipRouteProtocol,
ipCidrRouteAge Gauge32,
ipCidrRouteNextHopAS InetAutonomousSystemNumber,
ipCidrRouteMetric1 Integer32,
ipCidrRouteMetric2 Integer32,
ipCidrRouteMetric3 Integer32,
ipCidrRouteMetric4 Integer32,
ipCidrRouteMetric5 Integer32,
ipCidrRouteStatus RowStatus
}

Config Example 11  RFC 4292 MIB definition - IpCidrRouteEntry

Interesting for the discovery process are:

ipCidrRouteType INTEGER {
other (1), -- not specified by this MIB
reject (2), -- route that discards traffic and
-- returns ICMP notification
local (3), -- local interface
remote (4), -- remote destination
blackhole(5) -- route that discards traffic
-- silently
}

Config Example 12 RFC 4292 MIB definition - ipCidrRouteType

ipCidrRouteType - Defines the type of the route

Note that local(3) refers to a route for which the next hop is the final destination;remote(4) refers to a route for which the next hop is not the final destination.

Routes that do not result in traffic forwarding or rejection should not be displayed, even if the implementation keeps them stored internally.

reject(2) refers to a route that, if matched, discards the message as unreachable and returns a notification(e.g., ICMP error) to the message sender.  This is used in some protocols as a means of correctly aggregating routes.

blackhole(5) refers to a route that, if matched, discards the message silently."

ipCidrRouteNextHop – Directly gives us the IP address of the next hop neighbor.

ipCidrRouteIfIndex - The ifIndex value that identifies the local interface

through which the next hop of this route should be reached.  A value of 0 is valid and represents the scenario where no interface is specified."

ipCidrRouteProto - The routing mechanism via which this route was learned.

Inclusion of values for gateway routing protocols is not intended to imply that hosts should support those protocols."

SYNTAX INTEGER {
other (1), -- not specified
local (2), -- local interface
netmgmt (3), -- static route
icmp (4), -- result of ICMP Redirect
-- the following are all dynamic
-- routing protocols
egp (5), -- Exterior Gateway Protocol
ggp (6), -- Gateway-Gateway Protocol
hello (7), -- FuzzBall HelloSpeak
rip (8), -- Berkeley RIP or RIP-II
isIs (9), -- Dual IS-IS
esIs (10), -- ISO 9542
ciscoIgrp (11), -- Cisco IGRP
bbnSpfIgp (12), -- BBN SPF IGP
ospf (13), -- Open Shortest Path First
bgp (14), -- Border Gateway Protocol
idpr (15), -- InterDomain Policy Routing
ciscoEigrp (16) -- Cisco EIGRP
}

Config Example 13 - RFC 4292 MIB definition - ipCidrRouteProto

ipCidrRouteNextHopAS -"The Autonomous System Number of the Next Hop.  The semantics of this object are determined by the routing-protocol specified in the route's ipCidrRouteProto value.  When this object is unknown or not relevant, its value should be set to zero."

Step 2 – Find the right MIBs

Ensure that the MIB file and all the MIB files from that it depends are in the mibs folder specified with –m option.

Step 3 – Modify discovery-parameters.xml

Modify the discovery parameters file (the file specified with –f option). Discovery config file contains several sections describing the device types. An example for cisco is given bellow:

 

<device type="CISCO" xslt="discovery-manager/conf/xslt/transformator.xslt">
<discovery-method name="NEXT_HOP">
ipRouteIfIndex,ipRouteNextHop
ipCidrRouteType, ipCidrRouteIfIndex, ipCidrRouteNextHop, ipCidrRouteProto, ipCidrRouteNextHopAS
</discovery-method>
</device>

Config Example 14  Config example - add NEXT HOP discovery method to DiscoveryParameters.xml

Step 4 - Review  raw-data xml output

If IpCidrRouteEntry  is supported by the device. There will be such an entries in the raw-data xml file.

<ipCidrRouteTable>
<ipCidrRouteEntry>
<instance>0.0.0.0.0.0.0.0.0.172.16.11.1</instance>
<index name="ipCidrRouteDest">0.0.0.0</index>
<index name="ipCidrRouteMask">0.0.0.0</index>
<index name="ipCidrRouteTos">0</index>
<index name="ipCidrRouteNextHop">172.16.11.1</index>
<ipCidrRouteNextHop>172.16.11.1</ipCidrRouteNextHop>
<ipCidrRouteIfIndex>0</ipCidrRouteIfIndex>
<ipCidrRouteType>4</ipCidrRouteType>
<ipCidrRouteProto>3</ipCidrRouteProto>
<ipCidrRouteNextHopAS>0</ipCidrRouteNextHopAS>
</ipCidrRouteEntry>
<ipCidrRouteEntry>
<instance>10.6.4.0.255.255.255.252.0.0.0.0.0</instance>
<index name="ipCidrRouteDest">10.6.4.0</index>
<index name="ipCidrRouteMask">255.255.255.252</index>
<index name="ipCidrRouteTos">0</index>
<index name="ipCidrRouteNextHop">0.0.0.0</index>
<ipCidrRouteNextHop>0.0.0.0</ipCidrRouteNextHop>
<ipCidrRouteIfIndex>6</ipCidrRouteIfIndex>
<ipCidrRouteType>3</ipCidrRouteType>
<ipCidrRouteProto>2</ipCidrRouteProto>
<ipCidrRouteNextHopAS>0</ipCidrRouteNextHopAS>
</ipCidrRouteEntry>

</ipCidrRouteTable>

Config Example 15 raw-data xml output from the next-hop discovery method

Step 5 - Transform raw-data to a structured device-xml

To process them further the new data has to be accomodated in transformator-xslt file for the specified device type. The transformator.xslt aims to build from the unstructured raw-data a structured hierarchical object oriented xml database file for each of the discovered devices. That means:

  • There is one device
  • It has certain number of parameters
  • It contains certain number of objects
  • Each object also has some parameters
  • And may contain other objects.

Figure 1 Device model generated from deviceXML through Altova XML Spy

 

Each interface has ifIndex so it will be most appropriate to place the neighbors from ipCidrRouteTable as objects under the interface objects.

Step 6 - Determine Neighbor objects and objects parameters

<object>
<name>{HOSTNAME}</name>
<objectType>Discovered Neighbor</objectType>
<parameters>
<parameter>
<name>Discovery Method</name>
<value>{c<IPCIDRROUTEPROTO>,}</value>
</parameter>
<parameter>
<name>Neighbor IP Address</name>
<value>{X.X.X.X}</value>
</parameter>
<parameter>
<name>Neighbor hostname</name>
<value>{HOSTNAME}</value>
</parameter>
<parameter>
<name>Neighbor Device Type</name>
<value>{DeviceType}</value>
</parameter>
</parameters>

</object>

Step 7 – Determine Neighbor objects and objects parameters

Next task is to prepare xpath query to determine which cidr next hop is determined under which interface. The best option is to open the raw data file and using something like Altova xml spy to try finding the correct xpath.

Altova

Figure 2 Xpathing Device XML

<xsl:variable name="ipCidrRouteTable" select="//root/iso/org/dod/internet/mgmt/mib-2/ip/ipForward/ipCidrRouteTable/ipCidrRouteEntry[ipCidrRouteIfIndex=$ifIndex]"/>

Since there might be many routes with different next hops pointing towards this interface will be usefull to get only the unique values of it.

 

<xsl:variable name="next-hop-ips" select="distinct-values($ipCidrRouteTable/ipCidrRouteEntry/ipCidrRouteNextHop)"/>
<!-- So this will extract us the ipCidrRouteEntry values for the interface with the current ifIndex-->
<!--Foreach nexthop we have to check if it is not 0.0.0.0 or a loopback-->

<xsl:for-each select="$next-hop-ips">
<xsl:variable name="next-hop-ip" select="."/>
<xsl:if test="$next-hop-ip!='0.0.0.0' and not(contains($next-hop-ip,'127.0.0'))">
<!-- And to determine its neighbor id (e.g try to get through snmp its sysName)-->
<xsl:variable name="neighID">
<xsl:call-template name="return-hostname">
<xsl:with-param name="hostname-unformated" select="SnmpGetNameForXslt:getName($next-hop-ip, $comm)"/>
</xsl:call-template>
</xsl:variable>
<!-- If we are able to connect to the remote device and to get the sysName we will use it as an object Name-->
<xsl:if test="$neighID!=$sysName">
<object>
<xsl:choose>
<xsl:when test="$neighID!=''">
<name><xsl:value-of select="$neighID"/></name>
</xsl:when>
<xsl:otherwise>
<!-- Else we will use the ip address-->
<name><xsl:value-of select="$next-hop-ip"/></name>
</xsl:otherwise>
</xsl:choose>
<!-- Object Type of Discovered Neighbors has always to be Discovered Neighbor-->
<objectType>Discovered Neighbor</objectType>
<parameters>
<!--Discovery Method c+Protocol Names (separated by a comma)-->
<parameter>
<name>Discovery Method</name>
<value>
<xsl:for-each select="distinct-values($ipCidrRouteTable//ipCidrRouteEntry[ipCidrRouteNextHop = $next-hop-ip]/ipCidrRouteProto)">
<xsl:call-template name="ipCidrProtocolResolver">
<xsl:with-param name="number"><xsl:value-of select="."/></xsl:with-param>
</xsl:call-template><xsl:text>,</xsl:text>
</xsl:for-each>
</value>
<!--Tempate that returns parameters neighbor IP, Neighbor Hostname and neighbor Device Type.
Later on the discovery process will pick up for discovery only those neighbors that has an nonempty hostanme fileld
and will form the requests based on the device type.-->

<xsl:call-template name="return-neighbor-params">
<xsl:with-param name="neighborIP" select="$next-hop-ip"/>
<xsl:with-param name="neighborHostname" select="$neighID"/>
<xsl:with-param name="comm" select="$comm"/>
</xsl:call-template>

</parameters>
</object>
</xsl:if>
</xsl:for-each>

iTopoManager

The main task in iTopoManager is to ensure that the new method is correctly displayed in iTopologyViewer.

Step 8 - Prepate state 2 Transformation

Process the discovery parameters in stage two xslt transformation (transformation that creates the graphml file that will be used by the topology viewer in order to display discovered network topology).

Step 9 - Device to Graphml transfromation

Graphml format

Graphml transformators create graphological xmls from device xmls. There is a lot of information in the network what a graphml file should contain. Among the important parameters are graph type (directed / undirected) key id (for node or for edge where id has to be the same as attr.name).

<graphml>
<graph edgedefault="directed">
<key id="hostname" for="node" attr.name="hostname" attr.type="string"/>
<key id="deviceModel" for="node" attr.name="deviceModel" attr.type="string"/>
<key id="deviceType" for="node" attr.name="deviceType" attr.type="string"/>
<key id="name" for="edge" attr.name="name" attr.type="string"/>
<key id="method" for="edge" attr.name="method" attr.type="string"/>
</graph>
</graphml>

 

Then the transformator has to create the nodes and the edges.

Step 10 - Prepare Nodes

Each node has a unique id for the current file and zero or mode key ids.

In each graphml there is one major node with a lot of IDs representing the current device and many other nodes representing device neighbors. A simple code example creating the node for the current device is presented bellow.

<node>
<xsl:attribute name="id"><xsl:value-of select="$nodeID"/></xsl:attribute>
<data key="hostname">
<xsl:value-of select="$nodeID"/>
</data>
<data key="deviceModel">
<xsl:value-of select="$deviceModel"/>
</data>
<data key="deviceType">
<xsl:value-of select="$deviceType"/>
</data>
<data key="deviceStatus">
<xsl:value-of select="$deviceStatus"/>
</data>
<data key="BGPLocalASInfo">
<xsl:value-of select="$BGPLocalASInfo"/>
</data>
<data key="ipv6Forwarding">
<xsl:value-of select="$ipv6Forwarding"/>
</data>
<data key="ManagementIPAddress">
<xsl:value-of select="$ManagementIPAddress"/>
</data>
<data key="site">
<xsl:value-of select="$siteID"/>
</data>
<data key="deviceInfo">
<xsl:text disable-output-escaping="yes">&lt;![CDATA[ &lt;html&gt;</xsl:text>
<xsl:text disable-output-escaping="yes">&lt;b&gt;Type: &lt;/b&gt;</xsl:text>
<xsl:value-of select="$deviceType"/>
<xsl:text disable-output-escaping="yes"> &lt;b&gt;Model:&lt;/b&gt; </xsl:text>
<xsl:value-of select="$deviceModel"/>
<xsl:text disable-output-escaping="yes"> &lt;b&gt;Site:&lt;/b&gt; </xsl:text>
<xsl:value-of select="$siteID"/>
<xsl:text disable-output-escaping="yes"> &lt;br&gt;&lt;b&gt;Mgmt IP address:&lt;/b&gt; </xsl:text> <xsl:value-of select="$ManagementIPAddress"/>
<xsl:text disable-output-escaping="yes">&lt;br/&gt;</xsl:text>
<xsl:text disable-output-escaping="yes"> &lt;b&gt;ipv6Forwarding:&lt;/b&gt; </xsl:text>
<xsl:value-of select="$ipv6Forwarding"/>
<xsl:text disable-output-escaping="yes"> &lt;b&gt;BGPLocalASInfo:&lt;/b&gt; </xsl:text>
<xsl:value-of select="$BGPLocalASInfo"/>
<xsl:text disable-output-escaping="yes">&lt;/html&gt;]]&gt;</xsl:text>
</data>
</node>

It is important to note the <deviceInfo/> tag. It has html format and is the information presented by TopologyViewer as a node tooltip. DeviceInfo might give information about discoveryNeighbors and methods through that they are discovered. So following that direction the new method has to be correctly displayed also in that tag.

The rest of the nodes represent device neighbors. They are created by the following simple simple row:

<node>
<xsl:attribute name="id"><xsl:value-of select="$neighID"/></xsl:attribute>
</node>

Step11 - Prepate Edges

Edges represent the links between the devices. Each edge has an id, source and target and may contain one or more data keys. Source and target has to be nodes part of the current graphml file. IDs have one distinct feature - they have to be unique for the current graphml file. In the case of undirected graphmls IDs have to be the same in the current and remote graphml. Therefore generating undirected graphml is far harder then the directed one.

Step 12 - Review the result

Finally the new method has to be could be represtented by edge tooltips. Currenly such a tooltip is supported by the directed graphml views.

Figure 3 Edge ToolTip

Conclusion

iTransformer is a flexible framework for network development. It provides means for network discovery, inventorization and display and configuration. The current guide provides information how to extend the current iDiscover discovery algorithm by adding new neighbor discovery methods.  The guide also explains how to process the data in order the new neighbors and the new method tooltips to be correctly displayed by TopologyViewer. If you are looking for informaiton about what features are supproted by iTransformer please review the user guide. If you need information about how to configure those features to work propperly refer to configuration guide and if you are brave enough and want to further develop iTransformer and to add new features please have a look at the developer’s guide.

Tags: