Developer's Guide

Developer's Guide

Wellcome

Welcome to the iTransformer Dev Guide! The Dev Guide provides a practical introduction to developing applications for iTransformer and documentation about major product features. It explores the concepts behind iTransformers and the framework for extending and testing the application.

The Dev Guide holds most of the documentation for the iTransformer's platform, except for reference material on the how to develop new iDiscover discovery methods. For this one specifications, go to the Discovery Methods Implementation Guide.

Introduction

This guide contains information about iTransformer development rational and highlights the interfaces that could be used for further product extension development.

What is iTransformer?

iTransformer is simple but powerfull framework for network transformation. It is able to discover, model, preview and finally transform any network from one state to another.

iTransformer framework consists of several major moduls:

  • iDiscover - Discover IP network infrastructure and uploads it into a xml based network inventory model. The data in the model is filled in through a flow of several information (document) xslt transformations.
  • iTopoManager - Use the iDiscover data model to preview and to transform the discovered network infrastructure. Currently iTopologyManger has several sub modules:
  • TopologyViewer - previews network topology in directed or undirected graph view modes.
  • ResourceManager - manages network protocol connection parameters. The resources are declared in xml inventory file. Each resource is associated with a set of parameters. The ResourceManagement allows dynamic resource selection based on a match between the resource parameters and the parameters that are associated with the network device.
  • ParameterFactory - able to select and combine various parameters from iDiscoverer Network Inventory data model, TopologyViewer graphmls file, ResourceManager and parameters entered through manual entry.
  • FulFillmentFactory - able to fulfill the network transformation process to use the parameters passed from the Parameter Factory and to transform the network from one state to the other. With other words able to implement vaious engines for communincation with network devices and to pass them various parameters.
  • RightClickHandler - used to implement varoius device right click handlers. The right click might invoke fulfillment factory template a third party application or just to display an html report.

Development Environment

iTransformer is developed mosly in java and xslt. The following packages are typically required in order iTransformer to be able to operate in runtime:

  • ant - Java based build tool. Please Note: Version 1.5 OR HIGHER is required
  • junit - testing package
  • Java 1.6.0_25 JRE (or later).
  • Jung
  • Snmp4j
  • Mibble
  • Apache Commons
  • Log4j
  • Saxon9b
  • Snmp4j-1.11.2
  • Apache commons net 3.0.1

Typically the packages are supplied with iTransformer distribution.

iDiscover

iDiscover is a software product for discovery of IP network infrastructures. It implements a basic discovery algorithm and defines interfaces for extension and customising of that algorithm. The extentions of discovery algorithm are in following apects:

  1. Discoverer and DiscovererFactory - Implementation of different discovery protocols. The initial implementation supports SNMP.
  2. DiscoveryHelper and DiscoveryHelperFactory - Helps the “discoverer” to discover the network through the right requests.
  3. DiscoveryManagerListener - Implementing different handlers for discovery events for storing the data obtained during discovery procedure. Currently there are handers that store the data into a object oriented xml data format and into graphml data format.

Discoverer interface

The Discoverer and its factory define an interface that allows the iDiscover to be extended in order to support different discovery protocols. This can be achieved by new implementations of the Discoverer interface. The initial implementation of the Discoverer interface is  com.discovery.discoverers.SnmpWalker. It is possible a new Discoverers to be added by implementation of new Discoverer interfaces and adding it to the DiscoverFactory. The current implementation of the Discoverer interface looks like this.

public interface Discoverer {

    String getDeviceType(Resource resource);

    String getDeviceName(Resource resource);

    RawDeviceData getRawDeviceData(Resource resource, String[] requestParamsList);

}

The new implementations of the Discoverer interface have to implement the following methods:

getDeviceType - determines the Device Type. Currently that is done by obtaining the SNMP sysdescription from the device and parsing for specific deviceType strings in it. An example for such are the vendor strings as Cisco, Huawei, Juniper, HP etc. As an input argument is required a Resource.

getDeviceName - determines the DeviceName. Currently that is done by obtaining the hostname string throuh snmp. Once the hostname string is obtained the domain part of it is stripped. For example is you have R1.company.com the method will return you just R1.

getRawDeviceData - is the main method of that interface. Basically if somebody wants to add new Discoverer interface supporting new protocol has to reimplement mostly that one. It receives as an input parameter a resouce and and a bunch of request parameters. The result is a raw device data. The raw device data is a byte stream.

DiscoveryHelper and DiscoveryHelperFactory

DiscoveryHelper and its factory define an interface that allows iDiscover to be extended in direction off urther customization and extension of discovery algorithm. That can be done by adding new implementations of DiscoveryHelper interface and its methods.

public interface DiscoveryHelper {

DiscoveredDeviceData parseDeviceRawData(RawDeviceData rawData, String[] discoveryTypes, Resource resource);

Device createDevice(DiscoveredDeviceType discoveredDeviceType);

String[] getRequestParams(String[] discoveryType);

boolean checkStopCriteria(Resource host);

}

parseDeviceRawData – responsible for parsing the rawDeviceData. Currenly the parse process happens in the following way. Once the rawData is obtained it has to be transformed in a structured device-data xml. That happens with the help of the deviceType and the xslt transformation file defined in discoveryParameters.xml for that specific devicetype. The resource is needed because during the xslt transformation process are invoked most of the get_methods specified into the discoverer interface. This way of work allows us during the transformation process to perform SNMP queries in order to determine the values of some OID objects. Typical example for this is to determine the name of the neighbor device under certain interface (typically we will know only the IP address of that neighbor).

The second important role of that method is to unmarshal the device-data to a DiscoveredDeviceData that contains a java object structure from which could be extracted the neighbors ready for discovery.

createDevice – creates a Device structure that represents a single node in the network. As an input it has the output of the parseDeviceRawData method. The rational behind that method is to return to the discoveryMain process the device only with the device reachable device Neighbors inside it.

getRequestParams – obtains the the request paramters to be used for quering single a node in the network.The requests parameters are defined per deviceType in the deviceParameters.xml configuraiton file.

checkStopCriteria – performs a stop criteria conformance check. The stopcriteria are defined in discoveryParameters.xml file. The idea behind them is to tame the discoverer process in certain network boundary.

The DiscoveryHelper objects are instantiated by an abstract factory that has common interface – DiscoveryHelperFactory.

public interface DiscoveryHelperFactory {

    DiscoveryHelper createDiscoveryHelper(String deviceType);

}

Currently there is one implementation of the DiscoveryHelperFactory named XmlDiscovertHelperFactory.

createDiscoveryHelper - creates a discoveryHelper for certain deviceType. Typically the discoveryHelper is needed by the discoveryProcess once certain device is designated for discovery. In that moment we know the name and the type of the type of the device (therefore we use snmp queries in the xslt transformation process) so knowing the deviceType we can determine the type of the request towards the device. Then this request is used for obtaining the raw-device data.

The Definition of the different discovery helper factories is done in the discoveryManager.xml configuration file. If there is a new DiscoveryHelperFactory it has to be added here.

<discovery-manager>

    <discovery-helper class="com.discovery.discoveryhelpers.xml.XmlDiscoveryHelperFactory">

        <parameters>

            <param name="fileName">discovery-manager/conf/xml/discoveryParameters.xml</param>

        </parameters>

    </discovery-helper>

………….

</discovery-manager>

DiscoveryListener

Discovery listner is an event handling class that is registerd in DiscoveryManager main process and listens for newly discovered device events. The design rational behid the evet mechanism is to achieve flexibility towards integrating iDiscover with various third party inventory systems.

public interface DiscoveryListener {

    void handleDevice(String deviceName, RawDeviceData rawData, DiscoveredDeviceType discoveredDeviceType, Resource resource);

}

handeDevice - is a method handler that is invoked by the discovery algorithm each time a new device is discovered.

Initially has been implemented the following handlers:

  1. com.discovery.discoverylisteners.DeviceFileLogger – logs device-data.xml files into the filesystem.
  2. com.discovery.discoverylisteners.TopologyDeviceLogger - java based logger that generates graphml files for visualization of this device into the network graph (not used).
  3. com.discovery.discoverylisteners.XmlTopologyDeviceLogger – xslt based logger that generates graphml giles for visualization of this device into t network graph.

DiscoveryListeners are registerd in iDiscover main process with the help of DiscoveryHelperFactory:

com.discovery.discoveryhelpers.xml.XmlDiscoveryHelperFactory

The factory config is part of the discoveryManager.xml. The standard way of adding a new listener is to create a new class that implements discoveryListener interface and to add the class to the configuraiton file.  

<discovery-manager>

……………

    <discovery-manager-listeners>

        <discovery-manager-listener class="com.discovery.discoverylisteners.DeviceFileLogger">

            <param name="path">devices_and_models/lab</param>

        </discovery-manager-listener>

        <discovery-manager-listener class="com.discovery.discoverylisteners.XmlTopologyDeviceLogger">

            <param name="xslt">discovery-manager/conf/xslt/transformator-undirected2.xslt</param>

            <param name="path">devices_and_models/lab/undirected</param>

        </discovery-manager-listener>

        <discovery-manager-listener class="com.discovery.discoverylisteners.XmlTopologyDeviceLogger">

            <param name="xslt">discovery-manager/conf/xslt/transformator-directed.xslt</param>

            <param name="path">devices_and_models/lab/directed</param>

        </discovery-manager-listener>

</discovery-manager-listeners>

</discovery-manager>

iDiscover Resource Management

iDiscover has an internal resource class that is used by the discoverer interface implementatation but also utilizes the ResourceManager module. Initially iDiscover has been using only its internal class. The algorithm works the following way. During process initiation are passed as an input parameter the initial IP address and the discoveryManager.xml file. Into discoveryManager.xml is declared discoveryResource.xml config file. From the intial IP address and the default (currently SNMP) management protocol parametersis formed a Discovery Resource that is used for the discovery of the initial device. Then for any other discovered device the procedure is repeted (the internal Resource is formed by the device IP address and a lookup in the resources defined in discoveryResource.xml config file).

The design rational behind having both resources is the integration between the initial internal resource management and the common resource management used throught iTransformer.

ResourceManager

ResourceManager - store network resources protocol connection parameters and provides mechanism for selecting suitable resouce.

Resource manager is implemented by one ResourceManager class

com.resourcemanager.ResourceManager.

ResourceManager could be constructed by two constructors.

    public ResourceManager(ResourcesType resource) {

        …

    }

The first one constructs ResourceManager object from com.resourcemanager.config.ResourcesType. An example is provided bellow.  

public void testFindDeviceGroup() throws IOException, JAXBException {

        String xml = FileUtils.readFileToString(new File("resource-manager/conf/xml/resource.xml"));

        InputStream is = new ByteArrayInputStream(xml.getBytes());

        ResourcesType deviceGroupsType = JaxbMarshalar.unmarshal(ResourcesType.class, is);

        ResourceManager resourceManager = new ResourceManager(deviceGroupsType);

        Map<String,String> params = new HashMap<String, String>();

        params.put("geoloc","Varna");

        ResourceType result = resourceManager.findResource(params);

        Assert.assertEquals( "device-from-viena", result.getName());

    }

The second constructor constructs ResourceManager from xml file.

    public ResourceManager(String fileName) throws IOException, JAXBException {

        …

    }

The file has to contain one or more ResourceType resources.  Each Resource has name and may contain one or more connection parameters idetified by it connection type. Connection paramters are associated with map of paramers and their values.

<resources>

    <resource name="default">

        <connection-params connection-type="ssh">

            <param name="username">user</param>

            <param name="password">pass</param>

            <param name="timeout">3000</param>

            <param name="retries">3</param>

        </connection-params>

    </resource>

    <resource name="device-from-Moscow">

        <param name="geoloc">Moscow</param>

        <connection-params connection-type="ssh">

            <param name="username">user</param>

            <param name="password">Pass</param>

            <param name="system-password">Pass123</param>

        </connection-params>

        <connection-params connection-type="telnet">

            <param name="username">user</param>

            <param name="password">Pass</param>

            <param name="system-password">Pass123</param>

        </connection-params>

    </resource>

</resources>

ResourceManager provides two methods for access to the resources - findResource and getResource.

FindResource provides functionality to find a resource based on a “best first” match. The “Best” match is based on resource parameters as geoloc in the example above. The “First” match is used if there are two or more resources that match on the same number of parameters and values. In that case is returned the first one.  

public ResourceType findResource(Map<String, String> deviceParams)

getResource allows each Resource to be accessed from external modules or applications by its name.

public ResourceType getResource(String resourceName)

 

Fulfillment & ParameterFactories

Figure 1 Parameter and Fulfillement factories

ParameterFactory

The ParameterFactory is used to load and combine various parameters from device-data xml files, graphmls files, ResourceManager and parameters entered through manual order entry. Once the parameters are gathered they could be send to the fulfillment factory or to any third party application.

com.parameterfactory.ParameterFactoryBuilder

Build ParameterFactories. It has a constructor that loads ParameterFactories and ParameterFactoriesElementTypes from a xml configuration file.

    <param-factory-element-types>

        <type name="manual" class="com.parameterfactory.impl.GUIParamFactoryElement"/>

        <type name="graphml" class="com.parameterfactory.impl.MapDerivedParamFactoryElement">

            <param name="contextKey">graphml</param>

        </type>

        <type name="resource" class="com.parameterfactory.impl.MapDerivedParamFactoryElement">

            <param name="contextKey">connection-params</param>

        </type>

        <type name="deviceData" class="com.parameterfactory.impl.XmlParamFactoryElement">

            <param name="contextXmlFileNameKey">xmlFileName</param>

        </type>

    </param-factory-element-types>

A typical parameter factory might look like this.

    <param-factory name="TEST123">

        <param-factory-element type="graphml">

            <param name="ManagementIPAddress"/>

            <param name="site"/>

        </param-factory-element>

        <param-factory-element type="manual">

            <param name="a"/>

            <param name="b"/>

        </param-factory-element>

        <param-factory-element type="deviceData">

            <param name="firstFreeInterface">//DiscoveredDevice/object[objectType='Discovery Interface'][1]/parameters/parameter[name='CableCut' and value ='NO']/../../name</param>

        </param-factory-element>

        <param-factory-element type="resource">

            <param name="username"/>

            <param name="password"/>

            <param name="enable-password"/>

        </param-factory-element>

    </param-factory>

ParameterFactoryBuilder has one public method that builds cetain parameter-factory.  The factory “factoryName” has to be previously loaded by the constructor from the xml configuration file.

public ParameterFactory buildParameterFactory(String factoryName) throws IllegalAccessException, InstantiationException, ClassNotFoundException {

…. Output omited

}

com.parameterfactory.ParameterFactory

List<ParameterFactoryElement> elements - Each parameter factory creates one or more elements. Each element has to implement ParameterFactoryElement interface. 

public Map<String, String> createParameters(Map<String, Object> context) - each ParameterFactoryElement has one or more parameters. This method creates parameters into each element part of the ParameterFactory.

com.parameterfactory.ParameterFactoryElement 

This interface has to be implemented by each ParameterFactoryElement . It has two methods.

void init(Map<String, String> config, Map<String, String> params);

 Init methods acts like constructor. This method is supposed to be invoked once in the ParameterFactoryElement lifecycle.

Map<String, String> config contains static configuration data for initializing particular factory element.

Map<String, String> params contains parameters that should be initialized. The value of this map contains information per each parameter that will be used to create value for this parameter.

Map<String, String> createParams(Map<String, Object> context, Map<String, String> currentParams) throws Exception

Create map of parameters using the configuration obtained from init method invocation.

CreateParameters can create only such parameters that names are already defined in the key set of the params received in the init method.

That method could be invoked several times. Opposite to that the Init method is invoked always once per ParameterFactoryElement lifecycle.

Map<String, Object> context contains dynamic configuration data that could support parameter creation this factory element

Map<String, String> currentParams - contains currently created params by the ParameterFactory. CurrentParams usually comes from the invocation of other ParameterFactoryElements.

Junit test cases that demonstrates the usage of parameter factory.

public class ParameterFactoryBuilderTestCase {

    @Test

    public void testBuildParameterFactory() throws Exception {

        ParameterFactoryBuilder builder = new ParameterFactoryBuilder("parameter-factory/conf/xml/param-factory.xml");

        ParameterFactory factory = builder.buildParameterFactory("TEST");

        Map<String, Object> context = new HashMap<String, Object>();

        Map<String, String> graphmlParams = new HashMap<String, String>();

        graphmlParams.put("parameter","test123");

        context.put("graphml", graphmlParams);

        Map<String, String> resource = new HashMap<String, String>();

        context.put("resource", resource);

        Map<String,String> params = factory.createParameters(context);

        System.out.println(params);

    }

}   

FulFillmentFactory

FulFillmentFactory - Is used to automate the network configuration process.  iTransformer developers could create new fulfillemnt factories by implementing the public interface Fulfillment.  

public interface Fulfilment {

    void fulfil(ResourceType resource, Map<String, String> parameters, Map<String, String> fulfilmentFactoryParams, Logger logger) throws IOException;

}                                                                                          

The interface gets as parameters:

Map<String, String> parameters - parameters that come from the parameter factory.

Map<String, String> fulfilmentFactoryParams - parameters that come from the fulfillmentfactories.

RightClickHandler

 

Figure 2 RightClickHandlers menu

RightClickHandler - is used to implement varoius methods that could be invoked by a right click on a device in TopologyViewer. The right click might invoke fulfillment factory template a third party application or just display a html report. Rightclick handlers are the defined as the  iTransformer extension point for  new functionalities.

Each new rightClickhandler has to implement the following interface.

      public interface RightClickHandler {

    <G> void handleRightClick(JFrame parent, String v, Map<String, String> graphMLParams,

                              Map<String, String> rightClickParams,                        

                              URL s) throws Exception, IOException, ClassNotFoundException, IllegalAccessException, InstantiationException;

}

JFrame parent - The main TopologyViewer frame.

String v - The name of the node on which the rightclick is invoked.

Map<String, String> graphMLParams - Map with graphMLParams for the current node.

Map<String, String> rightClickParams - Map with rightClickParams specified in topologyviewer xml config file.

 

<rightClickItem name="Device Neighbors" handlerClass="com.topolgyviewer.rightclick.impl.XsltReportCreator">

                    <param name="xsl_transformator">rightclick/conf/xslt/deviceNeighbors.xslt</param>

                    <param name="table_transformator">rightclick/conf/xslt/table_creator.xslt</param>

 </rightClickItem>

URL s - URL to device XML files generated by iDiscover.

An example implentation XsltReportCreator (used to create and invoke XML reports) is presented bellow.

public class XsltReportCreator implements RightClickHandler {

     public <G> void handleRightClick(JFrame parent, String v,

                                     Map<String, String> graphMLParams,

                                     Map<String, String> rightClickParams,

                                     URL deviceDataXmlFileName) throws Exception {

…. Output ommited

}

     }

 

 

Tags: 

Subscribe to RSS - Developer&#039;s Guide