You are viewing [info]netcooler's journal

Previous 10

Dec. 29th, 2008

How to take network trace on HP-UX

Start logging to raw file:

#nettl -tn all -e ns_ls_ip -size 1024 -tracemax 99999 -f /tmp/raw0


Stop logging:

#nettl -tf -e all


To see captured packets in raw file:

#netfmt -N -n -l -c <filterfile> -f /tmp/raw0*


Examples of filterfile:

filter ip_saddr 192.0.0.1
filter tcp_sport 23
filter tcp_dport 23
filter udp_dport 162
filter udp_sport 162


To see messages in realtime:

#netfmt -N -n -l -1 -F -T -c <filterfile> -f /tmp/raw0*


-F - works as unix tail command
-T - display timestamp of packets
-1 - output by 1 line

To see messages in realtime without creating raw file:

#nettl -tn all -e ns_ls_ip | netfmt -N -n -l -1 -F -T -c <filterfile>
Tags: ,

Oct. 14th, 2008

Quick run of Netcool/Omnibus TSM for TL1

Omnibus 7.1, TSM for TL1 5.3

There are 3 places on the IBM POC site where it's possible to download TSM for TL1. In on of the places there is license file.

Installation quite straight - just run setup and say yes to everything. (choose to use buildin mysql database)

Then we need to config license.

Since we have license file for flexlm - then we need to put license file on the flexlm license server and set property licenseManager in netcooltn.properties file to port@host of flexlm server

By the way all config files for TL1 resides in <tl1 install dir>/config, although in docs we have different places described.
Here we have files:

  • props file, (omnibus)

  • rules file (omnibus)

  • tsm.xml file (TL1 config)

In tsm.xml we need to changes properties for user to connect to objectserver:

- jdbcuser (root default)
- jdbcpwd (null default)

Don't forget to specify enviroment variable as LD_LIBRARY_PATH=/opt/netcol/platform/<platform>/lib

And then we can run everything:

cd to <tl1 install dir>/bin

1. ObjectServer must be running
2. start_mysql.sh  (to run mysql)
3. nco_t_tl1


To config NEs we need to connect as client to TL1 with Pilot. Just put browser on <tl1 hostname>:5960 and enter login/password of ObjectServer user. We got interface where we can add/delere NEs to monitor.

Then we need to define all our NEs for monitoring. This is different topic.

Sep. 6th, 2008

RapidInsight for Netcool (RI4NC)

RapidInsight is an integration and automation solution for operationsmanagement, and typically used to integrate data from disparatemanagement systems and present to users through a simple, unified,consistent web based user interface.

In today's organizations, managing the IT infrastructure is a complexundertaking. Most organizations use number of tools for differentmanagement disciplines and for different technologies, yet the need tomanage the infrastructure holistically is apparent. RapidInsightprovides capabilities to consolidate data from disparate systems andmake it available to users through simple, consistent easy to use webbased interface to faciliate management processes.

RapidInsight is built upon standard web technologies therefore easilycustomizable and extensible without requiring proprietary know-how.

RapidInsight for Netcool provides an innovative web based interface to work with Netcool events:

  • an active (read/write) interface (AJAX) without no activex or java applets
  • the interface is designed to work with high volume of events with ease
  • easyto use google search like interface that enables users to create andsave queries to filter events by point and click, without requiringprogramming knowledge.
  • data from the server side is exposedover web api (XML/JSON), enabling development of alternative/additionaluser interfaces easily.
  • same interface is used to access historical events as well as active events
  • can be used to access historical events as well as active events, since events are not stored in memory.


source
Tags:

Oct. 30th, 2007

Groovy4Netcool script example: updating an event field

In the previous post, I'd announced the Groovy4Netcool project.In this follow up post, I'll go through a simple script as an exampleto demonstrate how it can be used. The objective is to work with anevent in the Netcool server, to get the value of a field and updatethat value.

First, I should highlight the files in this project are NOT required to use groovy to work
withNetcool server. Groovy language has inherent support to work withdatabase via jdbc. The files in this project provide an infrastructureto make it easier to work with Netcool so that one does not have toknow or deal with JDBC and SQL,
although they are available.

 

First, I need to set the connection parameters for the Netcool server in the NetcoolDataSource.groovy file.

static def datasourceParams = [

"netcool": ["url":"jdbc:sybase:Tds:host:port?LITERAL_PARAMS=true", "username":"nameoftheuser", "password":"passwordoftheuser"]

];

Now we can start writing our script. In our script we will use the ncEvent class (defined in ncEvent.groovy) file.

myEvent = new ncEvent()

The events in the Netcool server can be identified using the Serial orthe Identifier fields of the event. I'll use the getEvent method of thencEvent class to get the event. getEvent method takes the Serial of theevent as a parameter.

myEvent.getEvent( 898)

If I had the Identifier of the event but not the Serial, I couldhave used the getSerial method to get the serial(myEvent.getSerial("Router1Down"))

The event fields are now accessible in the script as properties ofthe myEvent object. myEvent.severity, myEvent.summary, etc. The eventfield names are used as all small letters. Let's print the identifier,summary and severity for the field.

println("Identifier: " + myEvent.identifier + " Summary: " + myEvent.summary)

We can also set the values of the event fields similarly.

myEvent.severity = 4

myEvent.summary = "summary set by example script"

At this point the properties of our event object has been updatedbut not the event in the Netcool server itself. To update the event inNetcool, I need to call the update method.

myEvent.update()

That's it! This script is available in the project as example1.groovy



source

Oct. 25th, 2007

Using groovy scripting to work with Netcool Omnibus

How do (or should) you interact/work/integrate with Netcool Omnibus servers?

If you work on systems integration, it's likely that you've run into this question at some point. There are number of options available depending on your needs. Here is a summary of the options that I could come up with:

Netcool suite components.

Not surprisingly, (since integration can be considered core competency) Netcool product suite includes products to faciliate integration with the Omnibus server. You can use "probes" to add (or update) events in Netcool, and can use "gateways" to take them out. Automations are used to work with events as well as "tools" that users can launch from the UI, etc. No need to talk much about these options as they are well known and documented. If you can do what you need using one of these options, you don't have a problem (not considering commercial aspects, etc. just from technical point of view)

Netcool Impact

Impact deserves a separate mention since it provides significant capabilities to work with netcool (omnibus) events as well as data from other sources. Impact provides a language of its own, to manipulate netcool events, enrich them with data from other sources, and many other things. It is safe to say that Impact can be the answer for most (netcool) integration/automation needs. The down side is that Impact is a complex, heavy product that is not easy to implement to say the least. It is also a big financial investment, hence it is not always a good option both from technical and commercial perspectives.

isql

Netcool provides a command line tool that allows executing queries on the database. isql is often used as a low tech tool since it can be called from shell/perl scripts, etc. and may be sufficient for the simple requirements.

Perl DBI/FreeTDS

Netcool components use Sybase libraries for communications so it is possible to use the database libraries (freetds is an open source JDBC library used instead of Sybase JDBC driver) to work with the Omnibus server. Given the flexibility and the power of Perl, this approach means you can do pretty much whatever you want once you get it to work. An open source tool, LooperNG, uses this method to integrate with Netcool.

The problem is, it is quite difficult to get it all up and running. It requires a highly skilled system admin to put the pieces together and there is not much information on how to get this done. Netcool Users Group mailing list is the best source of information on how to get the Perl DBI working as it is for anything related to Netcool.

Despite the difficulties, it's still a good option if you want to work with a non-proprietary powerful scripting language, particularly if you're a skilled Perl programmer.

Java/JDBC

As stated above, since Netcool components use Sybase libraries, it is possible to work with Omnibus server Sybase JDBC driver (some of the native Netcool components use this method). If you're a java programmer, this may be the way to go, keeping in mind that Omnibus may use Sybase libraries to communicate but the database itself is NOT Sybase but a proprietary, in-memory database developed by IBM (MicroMuse), therefore some things may not work as expected.
There is not much information on how to set this up either. One has to go through Sybase JDBC documentation as well as Netcool manuals, etc. to get it working.

As I've stated, each of these methods/tools has pros and cons. I've often found myself frustrated with the shortcomings of these approaches and wanted to combine the positive aspects some of these methods. Enter Groovy4Netcool:

  • easy to use, non-proprietary, powerful, dynamic scripting language (Groovy)
  • platform independent (works on any OS with a JVM)
  • easy to install and configure, no code compilation, dependencies, etc.
  • has some documentation, examples, etc.


Groovy is a dynamic scripting language for Java Virtual Machine (JVM). We use Groovy heavily in our products since we believe JVM as the platform combined with dynamic scripting is the killer combination. In a sense, Groovy brings the pragmatic, dynamic approach of Perl into the Java platform. I can that my head hurts when I look at java code for more than couple of minutes; but as a system integrator (and not a developer) who can handle some degree of scripting, I find Groovy manageable.


To me, the key is the JVM. Having access to all java libraries, and not having to worry about the operating system, compiling code, hunting down dependencies, etc. is wonderful. Most computers already do have a JRE, and one can get groovy4Netcool up and running in minutes, no system admin blackbelt is needed here!

We've set up an open source project hosted on Google projects, called Groovy4Netcool to share what we have. Groovy4Netcool can be considered more of a systems integration project than a typical software project. It is not a "product". The objective is to make it easier to use groovy to work with Netcool Omnibus servers by providing instructions and examples, as well some infrastructure code.

Hope you find it useful.



source

Jul. 19th, 2007

Some enrichment technique in OMNIbus

On 18 Jul 2007, at 16:30, Scott Ware wrote:

I did install DBD-Sybase, and FreeTDS and everythingwent will with that, but here's what I ended up doing to get it working:
I set up a temporal trigger to run the below scriptevery 10 minutes. For every message that we get from our switches thatis in the AlertGroup 'LINK-ERROR', this script telnets into the device,grabs the description on the given port that is reporting the errors,and then updates the summary to include the description of the port inparenthesis at the end of the summary.


#!/usr/local/bin/perl
use Net::Telnet;
($device, $interface, $serial, $summary) = @ARGV;
$args = $#ARGV + 1;
if ($args < 4) {
  print "Usage: int_descr.pl <device> <interface><serial> <summary>\n";
}
else { 
  &telnet($device, $interface, $serial, $summary);
}

sub telnet {
  $user = "xxxxxx"; $pass = "xxxxxx";
  $db_user = "xxxxxx";
  $db_pass = "xxxxxx";
  $telnet = new Net::Telnet( Prompt => '/.*[>#]\s*$/', Timeout => 10);
  $telnet->open($device);
  $telnet->errmode("return");
  $telnet->login($user, $pass);
  @descr = $telnet->cmd("show run interface ".$interface ." | include description");
  foreach $line (@descr) {
    chomp($line);
    if ($line eq "") {
      $line = "No description";
      $new_summary = $summary ." (".$line .")";
      &update_netcool($new_summary,$serial);
    } else { 
      $line =~ s/.*description\s+//g;
      $new_summary = $summary ." (".$line .")";
      &update_netcool($new_summary,$serial);
    }
  }
  $telnet->close;
}

sub update_netcool {
`/opt/netcool/omnibus/bin/nco_sql -user xxxxxx -password xxxxxx << EOF
 update alerts.status set Summary = '$new_summary' where Serial = $serial; go EOF`;
}

source

Jun. 25th, 2007

What You Need to Know (WYNTK) on Tivoli Business Service Manager (TBSM) 4.1: Services

Service Instances, Instances or just Services as they are now calledare the unique instantiations of templates within TBSM 4.1. Servicesare the “living things” within TBSM 4.1 and enable us to create theunique relationships and linkages to the underlying data sources.

Building Services starts with naming. Just like I mentioned for yourTemplates, you should develop a functional and unique naming schema forServices. Each Service has a specific Service Name and Display Namevariable. Your naming schema and approach for Service Name shouldcapture the key attributes that enable you and/or your administratorsto understand what it is. Consider adding a unique enumeration on theend of your Service name “APPServer01-ERP-NYC-028-01” as a key to tieback into your Instance business service/application (ERP),documentation (028) and versioning information (01).

Setting your Display Name can be challenging. There is enough realestate for 22 characters. Anything beyond that and auto-logic willcondense your Display Name into the 14 characters followed by anellipsis (…) and then the last five characters. You will need to managethis or you will quickly see that your Service names are garbled in thetreeview or that they extend and bleed over on canvas widgets. Ifnon-technical or business groups will be included in your audienceprofile, consider Display Names that will make sense to them if youplan to show them portions of the Service Tree, Service Model orScorecards. You can change this behavior in the RAD_sla.props file bychanging the impact.sla.tree.displayExp parameter. A fix is planned toallow auto sizing of the Display Name to occur if the Service Treeframe is resized and there is room in the column to show the entireDisplay Name.

Note: Don’t set your Display Name to ‘01234567890123456789’ as Ijust did this to test how long this field could be and got the ServiceTree to throw an exception and corrupt the Service Tree display.There’s a good test scenario for development now. :-) I had to manually delete the Service from the database table to fix this.

You will really want to think about establishing a formal schema andapproach to this, especially if your IT organization lacks a functionalor logical naming scheme for their infrastructure components. Don’t beintimidated by doing this. You may have tens of thousands of Servicesbut if you use the capabilities of the product you can build servicesand your naming structure automatically. Remember, if you click on theDelete Service icon, you’re presented with a list of all Services. Howwill you remember or identify the “right” one to delete when things maybe named similarly?

Template assignment comes next when building Services. It is thefirst of three steps for status propagation to occur from the lowerlevels of your Service Model to the highest. At the very least, aPrimary Template must be assigned to the Service. The Primary Templateis significant because the Icon and SLA rules are limited to thistemplate. The Primary Template also overrides when additionalproperties are configured with multiple templates having differentvalues of one property name as the Service will use the value of thePrimary Template.

Services can have multiple templates assigned. This means that theService will inherit the behaviors of all assigned templates. All ofthe status and ESDA rules on those assigned templates will be assignedto the Service. There is no limit to the number of templates that canbe assigned to a Service.

The next step required for status propagation is establishing thenecessary Identification Fields required for making this Serviceunique. We bring a Service to life using the Identification Fields toestablish unique matching criteria with data sources. TheIdentification Fields are like the “where” clauses of an SQL query andhelp us to narrow things down for the Service. We’re looking forspecific content from the fields of the event or database tables here.If you don’t have it in there, it’s going to be hard to match and makethe Service unique.

In an example for event based Incoming Status Rules, theIdentification Fields are used to determine which Services an eventmatches. Initially, Incoming Status Rule processing determines if theevent matches the filter in the incoming status rule. If so, it thenchecks all the Services of the template containing the Incoming StatusRule to see if their Identification Fields match the event. A trick fordealing with “less than optimal” events is to use multipleIdentification Fields. For instance, if you can’t expect to get a fullyqualified host name every time in your event, you can add oneIdentification Field for the full host name, another with the shorthost name and another to deal with case variables. MultipleIdentification Fields are treated as ID1 ‘OR’ ID2 ‘OR’ ID3, etc.

There’s no limit to the number of Identification Fields for aService. No efficiencies are gained using a specific type ofIdentification Field (VARCHAR, STRING, INT, etc.) and there isn’t anorder of precedence in how the matching algorithm will search forIdentification Field matches. Everything is done in memory and is veryoptimized. However, the fewer number of different identification setsyou configure, the less matching needs to be done. This goes back tothe importance of having purpose built normalized events which containspecific fields that enable efficient Identification Field matching tooccur.

The first two components are all that’s required for status to berendered for a specific Service without dependencies. It’s more thanlikely that you will be creating Service Models that make heavy use ofdependency relationships. The last step required for status propagationfrom within these models is to establish Dependencies for Services.

Dependency relationships can be one to one, one to many and even“circular” in nature (Service depends on itself). EstablishingDependencies accomplishes two things within the Service Model. First,the visual relationship is established within the Service Viewer.Adding a Dependency to other Services will draw the line with the arrowpointing towards the dependent Service. Services dependent onthemselves have an arrow that loops back around and points to theoriginal Service. There are some best practices for the visualizationof dependencies that I’ll touch on in a later WYNTK. These deal withhow processes, transactions and “flows” should be visualized to bestpresent their dependencies and relationships.

The second thing that happens upon establishing Service Dependenciesis that status can propagate from a child to its parent. For this tohappen, the parent Service must belong to a Template that contains arule that depends on a rule in a Template to which the child Servicebelongs. If a Service has multiple dependencies, the status of allchildren will propagate to the parent Service. The same rule applieshere; there must be a dependency rule on the Template level in additionto the Service dependencies.

Study the templates and instances used to build the integration withTADDM using the XML Toolkit and the Service Component Repository to geta good feel of how things get built dynamically via this integration.The ESDA rules used to build the Services (read the policy within theESDA configuration) for a good deep dive into the possibilities ofcreating Services dynamically.

You should develop a documentation methodology for capturing keyService information, why you created the Service, version number,associated templates, dependencies, security, ISM configurations, etc.Being able to remember why certain Services were created, who hasaccess to them, what their dependencies are will pay huge dividends asyour solution matures. This documentation will become a core componentof your documentation portfolio for each Business Service Managementsolution you develop in TBSM 4.1.

I have some ideas for how all of this documentation could becreated automatically, stored in a database and to have a nice web 2.0look and feel on top. If you’d like to work on such a project, pleaselet me know.

You should develop a versioning approach for managing changes toyour Services. This could be as simple as exporting Serviceconfigurations from TBSM 4.1, parsing into Service units and storingtheir configurations in CVS or Subversion. If you’re making a lot ofchanges or have a primary/failover environment, you’re going to want tothink of this stuff to keep everything in sync.

You should develop testing use cases for each Service and Servicehierarchy. These should include sample data that can be passed throughthe system(s) to test each state, rule or expected metric from eachService and Service hierarchy. You should have test files which containevent profiles, database tables that change key values being watched inrules and enough historical sample data as needed to run through anyscenarios needed for any complex calculations performed or trendingneeds.

Scalability has been tested to at least upwards of 100K uniqueServices that I’m aware of in varying sized (depth/width) servicemodels. There is a fairly linear growth curve in JVM memory required tosupport Services. The more Services you have, the more memory consumedto persist those Services in memory. A performance and scalabilitywhitepaper is planned in the near future. Your IBM Tivoli accountmanager or technical team should be able to get this for you. I do knowthat the testing and benchmarking group needs real world scenarios fromthe field to help improve their testing processes and feedback theyprovide into development.

I’d enjoy hearing your feedback on Services, how you are creatingand using Services and opportunities where we can improve the Serviceconcept and Service definition and administration features in thefuture. Would a more graphical or visual interface make sense fordefining Services? Would a drag and drop interface make establishingService dependencies easier? Would a wizard or more logic for settingIdentification Fields be useful? How about Visio import or BPEL orother XML type import?



source
Tags: ,

May. 14th, 2007

How to swap bytes in OMNIbus rules


$port = 36895

# Swap bytes in $port -> $port_new

$port_lo = int( $port / 256 )
$port_hi = $port % 256
$port_new = ( 256 * $port_hi ) + $port_lo


source
Tags:

May. 7th, 2007

Impact & ODBC DSA for Solaris

If we need to get access to Oracle database on Solaris via ODBC we need to install ODBC.

There are some options to do this. One of the ways is to use unixODBC

And also we need odbc driver for Oracle. I found two possibilities:

DataDirect Connect® for ODBC
Easysoft ODBC-Oracle Driver

Both are NOT for free.
Tags: ,

FreeTDS 0.64 + PHP 5.2.1 - installation guide

Apache Installation

Download the latest source from http://httpd.apache.org

root@cicostc # gunzip httpd-2.2.4.tar.gz
root@cicostc # tar xvf httpd-2.2.4.tar
root@cicostc # cd httpd-2.2.4
root@cicostc#./configure --enable-so --with-layout=Apache --enable-ssl --with-ssl=/usr/local/ssl
root@cicostc # make
root@cicostc # make install
root@cicostc # cd /usr/local/apache2/
root@cicostc # vi conf/httpd.conf


Start the Apache Server


/usr/local/apache2/bin
root@cicostc # ./apachectl start


Open a browser and check that Apache is running, http://localhost/

You should see:


It works!


FreeTDS Installation

Download the latest source from http://www.freetds.org/


root@cicostc # tar xvf freetds-stable\[1\].tar
root@cicostc # cd freetds-0.64


Put C-Compiler in the Path


root@cicostc # PATH=$PATH:/opt/gcc/bin/
root@cicostc # export PATH
root@cicostc # ./configure --prefix=/usr/local/freetds
root@cicostc # make
root@cicostc # make install


Add the Sybase environment variables:


# Variable needed for isql binary
SYBASE=/opt/netcool/omnibus/platform/solaris2; export SYBASE

LD_LIBRARY_PATH=$OMNIHOME/platform/solaris2/lib ; export LD_LIBRARY_PATH


Edit the freetds configuration file to contain an entry for the
InfoServer, comment out the textsize property


root@cicostc # pwd
/usr/local/freetds/etc

root@cicostc # vi freetds.conf

[CIC_POS]
host = 10.50.17.1
port = 4100
tds version = 5.0

#text size = 64512

root@cicostc # pwd
/usr/local/freetds/bin

root@cicostc # ./tsql -H cicostc -p 4100 -S CIC_POS -U root -P root
locale is "C"
locale charset is "646"
1> select count(*) from alerts.status;
2> go
COUNT( * )
1392


PHP Installation

Add libxml2 support, download the version from ftp://xmlsoft.org/


root@cicostc # gunzip libxml2-2.6.11.tar.gz
root@cicostc # tar xvf libxml2-2.6.11.tar
root@cicostc # cd libxml2-2.6.11
root@cicostc # ./configure
root@cicostc # make
root@cicostc # make install


Download the product version from http://www.php.net/


root@cicostc # gunzip php-5.2.0.tar.gz
root@cicostc # tar xvf php-5.2.0.tar
root@cicostc # cd php-5.2.0
root@cicostc #./configure --prefix=/usr/local/php --with-sybase-ct=/usr/local/freetds --with-apxs2=/usr/local/apache2/bin/apxs
root@cicostc # make
root@cicostc # make install


Edit the AddType section and LoadModule of the apache configuration to
contain the below. Restart apache to load php.


root@cicostc # cp php.ini-dist /usr/local/lib/php.ini
root@cicostc # vi /usr/local/apache2/conf/httpd.conf

AddType application/x-httpd-php .php
LoadModule php5_module modules/libphp5.so

root@cicostc # cd /usr/local/apache2/bin/
root@cicostc # ./apachectl restart

root@cicostc # pwd
/usr/local/apache2/htdocs

root@cicostc # touch phpinfo.php
root@cicostc # vi phpinfo.php




Test that the php module has loaded, create a simple script and place
it in the document root of the server.

Open the browser in the document root e.g.

http://10.50.17.1/phpinfo.php

You should get the table


Use the Transact-SQL command line tool to test your connection to the
InfoServer.

source

PS: We need to patch PHP or FreeTDS to handle the issue where field names & varchar values have NULL terminators

Previous 10