OpsMgr Database Hygiene: SCC Health Check Reports Management Pack by Oskar Landman & Pete Zerger

This version is a follow up based on feedback on the first version.

Apart from more knowledge and links added we have also wrapped up almost twice the amount of reports in this version!

Version 2

A quick review on all reports in this version: New additions!

Report Name	Queries Database
Agents - Agent Hotfix Report (OM)	OperationsManager
Agents - Agents Missing Hotfix Report (OM)	OperationsManager
Agents - Down Agents (OM)	OperationsManager
Alerts - Alerts Closed by Specific User (DW)	OperationsManagerDW
Alerts - Alerts Closed by User [Count] (DW)	OperationsManagerDW
Alerts - Alerts Closed by UserID (DW)	OperationsManagerDW
Alerts - Number of Alerts per Day (OM)	OperationsManager
Alerts - Top 20 Alerts By Alert Count (OM)	OperationsManager
Alerts - Top 20 Alerts By Repeat Count (OM)	OperationsManager
Config Churn - Discoveries Last 24 Hours (DW)	OperationsManagerDW
Config Churn - Modified Properties Details Last 24 Hours (DW)	OperationsManagerDW
Events - All Events Count By Last 7 Days (OM)	OperationsManager
Events - Most Common Events by Number and Publisher (OM)	OperationsManager
Events - Top 20 Computers Generating the Most Events (OM)	OperationsManager
Misc - Groups Report (OM)	OperationsManager
Misc - Infrastructure Overview (OM)	OperationsManager
Misc - Management Packs (OM)	OperationsManager
Misc - Operational and Datawarehouse Usage Report (OM) - (DW)	Both
Misc - Outage and Maintenance Report (DW)	OperationsManagerDW
Misc - Run As Profiles (OM)	OperationsManager
Performance - Performance Inserts Per Day (OM)	OperationsManager
Performance - Top 20 Computers By Perf Count (OM)	OperationsManager
Performance - Top 20 Performance Insertions By Perf (OM)	OperationsManager
Performance - Top Performance Baseline Generating Rules (OM)	OperationsManager
State - Noisiest Monitors (OM)	OperationsManager
State - Old State Changes Not Groomed (OM)	OperationsManager
State - State Changes Per Day (OM)	OperationsManager

We have also changed the documentation a little because some people reported authentication problems when running the Ops DB reports.

Feedback

Again the idea behind this MP is a real community MP. If you have feedback or additions please contact Oskar Landman or Pete Zerger and we are happy to add them to version 3! Of course your additions will be recognized by the community it the way of special credits!

The feedback can be in several ways of adding knowledge like links to good articles to fancy SQL queries to retrieve data out of the DB. Also if you feel we are missing crucial reports and don’t have the sql query to retrieve the data let us know we always  like a challenge ;-)

Besides this we have also received rules and monitors to monitor the health of your environment more closely. We are currently looking if we will fit this in version 3 or make an additional MP accompanied by the reporting MP.

Future

The future of this MP will depend on the feedback and additions received. Although I can already tell you version 3 will be released and in here we will consolidate the reports to bring down the number of reports which will make it faster to check up on your environment! Because many of them are related.

Special Credits

Special thanks to  Kevin Holman for his great feedback and as a query resource ;-)

And of course Pete Zerger for his reports and query additions! Always fun and looking forward to version 3! maybe we can do this one in German ;-)

The writers of this MP are not responsible for any problems derived from the MP although the MP has been tested on different installations and there are no known issues. There is always a possibility things are different in your environment so as always use the documentation and make sure that you know what you are doing.

The MP can be downloaded at the MP catalog.

Have Fun and feel free to comment or send in your feedback!

Most of the times it comes down to running SQL queries directly on the databases to gather this information. After gathering the information you can retrieve the information widely spread on the Internet and start tuning over enthusiastic rules and monitors to keep your environment nice and tidy.

This Management Pack is written to make your life more easy!

The story behind the MP

When we first started talking about this MP the question “Should we put it into the open?” was one of the first questions we needed to answer.

Because reporting on data from the Ops DB is not supported by Microsoft. But with some effort you will soon find lots of queries to retrieve the data from the Ops DB by running queries directly on the Ops DB.

There are tons of them around on the internet and this MP may be a guidance on how you should use them in a need reporting format.

Pete and I started of with building some custom reports and use commonly used queries to retrieve the data, mostly derived from his series. Based on this I expanded the layout and settings to proper reports.

Next I started to gather some more useful queries from the famous blog byKevin Holman which has a long list of really useful queries and postings on performance and tuning.

Now I wrapped up all queries we think are useful gave the reports a knowledge addition and added them to a Management pack called SCC Health Check Management Pack!!

Community MP

The management pack is included with a small readme because you need to create a custom shared datasource to be able to connect to the Operational Database.The idea behind this Management Pack is it will be a community MP with ongoing development.

What does this mean?

Well as you know there are many articles written nowadays on SCOM and how to get it and keep it running smooth. The difficult part is most of the articles are scattered all over different blogs.

The idea behind this MP is to run the reports and read the knowledge accompanied by the report which should include a short description of the report and mostly important hyper links to articles explaining why the data is important and how you should interpret it.

The reports and knowledge links should be included in this MP by SCOM experts with their findings and real life recommendations!

Come to think of it this is one of the benefits for using management packs developed with the knowledge of experts in the first place!  ;-)

This is just the first version which has some knowledge but along the way we will extend the reports and knowledge based on your feedback!

Feedback

Because we want to have a real life community MP it is crucial for people to deliver their feedback and additions to this MP. All feedback will be reviewed by the authors Oskar Landman and Pete Zerger and if relevant will be added to the Next version of the SSC Health Check Management Pack. This way this MP can be a real benefit for running a smooth SCOM environment.

The SCC Health Check Management Pack

The MP contains 16 reports which are required to run on the Datawarehouse or the Operational Database. There is documentation accompanied how to set up the solution and import the MP.

Please first read the documentation otherwise the MP will fail…..as you already know as SCOM Experts ;-)

You need to create a custom Shared Datasource on your report sever to be able to run the reports on the Operational Database.

After creating the Shared Datasource on your reporting server you can add the Management Pack! Doing any sooner and the import of reports will fail! Which you should already know since as a SCOM expert you always read the documentation before adding any MP………….right???

Here are the reports currently in the MP:

The writers of this MP are not responsible for any problems derived from the MP although the MP has been tested on different installations and there are no known issues. There is always a possibility things are different in your environment so as always use the documentation and make sure that you know what you are doing.

The MP can be downloaded at the MP catalog.

Have Fun and feel free to comment or send in your feedback!

Basics

How to Setup SNMP on both SCOM and the Network device or Windows Server.

Authoring Console:

How to create a custom class based on Network Device and how to discover it.

The example MP is attached at the bottom of this post.

SNMP overall

This part is where most of the problems start. Because of wrongly configured SNMP Settings the devices never get discovered in the first place.

First you need to make sure SNMP is setup correctly on both the SCOM Management server and the Windows Server or Device you want to discover. Without the correct setup we are not going to discover anything ;-)

First some details on SNMP:

Wikipedia - SNMP operates in the Application Layer of the Internet Protocol Suite (Layer 7 of the OSI model). The SNMP agent receives requests on UDP port 161. The manager may send requests from any available source port to port 161 in the agent. The agent response will be sent back to the source port on the manager. The manager receives notifications (Traps and Inform Requests) on port 162. The agent may generate notifications from any available port

Basically our SCOM management server will act as a SNMP Manager and our device is going to act as a SNMP agent.

SNMP in detail on TechNet

SNMP on Windows 2003 from the Knowledgebase

Maybe obvious but always make sure you can communicate by port UDP 161 and 162 between the SCOM Server and the Windows Server or Device.

SNMP Setup SCOM Server 

For easy understanding we are going to use the example above.

On the SCOM Server SCOM2007.mydomain.local

First we need to install / add the SNMP components to SCOM2007.mydomain.local.

Windows 2003		Windows 2008
	Make sure you have access to the Windows 2003 Installation Source. (i386)		To install SNMP on Windows 2008 you will need to use server manager and add the SNMP Service feature.

After the installation you will have the following services on the SCOM Server:

You can leave the default settings.

SNMP Managed Devices

Network Device / other SNMP Device

When you want to add a network device you will also need to configure SNMP on the Device. How to do this will differ by device, most of the times you can find help on changing these settings in the manual of your device.

When you know how to you will simply need to add the SCOM Server, SCOM2007.mydomain.local or 10.10.10.10 and its community name scomtestsnmp on the device to be able to discover it by the use of SNMP.

*****TIP! Always make sure you know the version of SNMP your device is using. SCOM 2007 R2 Supports both SNMPv1 and SNMPv2. SNMPv3 is not supported in the current version we will have to wait until VNext. When running a Network Device discovery you need to specify the SNMP Version make sure you use the correct one otherwise discovery will fail.

Windows Server by SNMP

Win2003.mydomain.local or 10.10.10.30

(Settings apply to both Windows 2003 as well as Windows 2008.)

*****TIP! When using 2008 always make sure the Windows Firewall has exceptions for SNMP Traffic.  

When you want to add a windows server to your SCOM environment by the use of SNMP. You should also install the SNMP Service like we did at the SCOM server.

After the Installation You need to Adjust the SNMP Settings to allow the SCOM Server,SCOM2007.mydomain.local, to communicate with the Windows Server,Win2003.mydomain.local.

Simply navigate to the SNMP Service run—>Services.msc\SNMP Service

Select the properties and navigate to the Security tab and add the SCOM Server IP Address or Hostname and add the Community name you want to use.

In our Example we are going to use:

Community String scomtestsnmp

SCOM Server IP is 10.10.10.10 or hostname SCOM2007.mydomain.local

Configuring these settings on the Windows server you want to discover by SNMP, Win2003.mydomain.local, allows the SCOM Server to talk SNMP to the Windows 2003 Server ! Without these settings the windows server will never be discovered as a SNMP Device…..

If you followed the above procedures you should be able to now discover your SNMP enabled devices. When discovering these devices you are able to use the basic monitoring rules default to SCOM to monitor your device.

Next we are going to use the authoring console to configure a custom class and how we can discover the class and any additional properties.

Like stated this one is based on a forum post / question. Therefore I will use the same Management Pack and approach.

The approach was to discover an Application running on Windows 2003 that uses it’s own MIB.

Because monitoring requires to send SNMP get requests to check for a state

Create Management Pack

Create a Empty management pack named: XYZ.Management.Pack

On the properties tab of the management pack make sure you have the correct mp references:

If they are not present please add them.

Create a class named MyApp - XYZ.Management.Pack.Class.MyApp set the base class to be Microsoft.SystemCenter.NetworkDevice.

Because the Base class is Microsoft.SystemCenter.NetworkDevice it will automatically inherit al the Network Device properties.

Add a new property called Version to the class and don’t forget the displayname ;-)

Create the Datasource for the Discovery

Create a Datasource named XYZ.Management.Pack.DataSource.Discovery.MyApp

Member Modules			ConfigurationSchema
Scheduler – System.Discovery.Scheduler	SNMPGetProbe – System.SNMPProbe	Mapper – System.Discovery.ClassSnapshotDataMapper

How the datasource works:

If you want to discover more properties you can add them to the SNMPprobe Module.

The easiest way is to copy the XML and change the OID you want to query.

The ISwriteAction will always be false since we wan to do a SNMP get and not a SET!

 Create Discovery Rule

Create a Custom discovery Rule called XYZ.Management.Pack.Discovery.MyApp

Target the discovery at Microsoft.SystemCenter.NetworkDevice

For Discovered Classes select the MyApp Class - XYZ.Management.Pack.Class.MyApp

At the configuration tab you need to browse for the datasource, XYZ.Management.Pack.DataSource.Discovery.MyApp, you just created and select it.

Next you need to add the variables we declared in the datasource. Interval - Remember the interval is in seconds 86400 = 1 day so the discovery runs daily. IP - Because we set the target to Microsoft.SystemCenter.NetworkDevice we can simply use it’s properties. Use the arrow to select the IP property of a Network device. CommunityString - Use the arrow to select CommunityString from the target property Network Device. *****TIP! Remember to first delete any values present in the Value box, like IP,Community string when selecting the arrow next to the box. Otherwise the value selected will be just added and you will get something like CommunityString$Target/Property[Type="NetLib!Microsoft.SystemCenter.NetworkDevice"]/CommunityString$ instead of $Target/Property[Type="NetLib!Microsoft.SystemCenter.NetworkDevice"]/CommunityString$

 Create Monitors / Rules links

After discovering our custom class for the application we are now able to create rules and monitors target to the class.

Since the creation of rules and monitors really depend on How you want to monitor your application or device here are links from basic to advanced on how to setup monitoring.

System Center Central

SNMP Series by Pete Zerger

SNMP Support of Devices by Pete Zerger

SNMP Probe Based Monitors by David Allen

SNMP Device simulator by Jalasoft

SNMP Management Pack Example - NetApp by Oskar Landman

Other

SNMP Monitoring by Kris Bash -  http://operatingquadrant.com/

Microsoft

How to create a SNMP Probe-Based Unit Monitor in Operations manager 2007

How to Create a Monitor for SNMP Traps in Operations manager 2007

How to Create a SNMP Performance Collection Rule in Operations manager 2007

How to Create a SNMP Trap Event Collection rule in Operations Manager 2007

How to Create a SNMP Event Collection Rule in Operations Manager 2007

How to Create an Alert-Generating SNMP-Trap-Based Rule in Operations manager 2007

Although the approaches may be the same for most steps in Visual Studio 2008 the reports created in Visual Studio are NOT backwards compatible with Sql Reporting Services 2005. Reports designed with visual studio 2008 can only be used if you are running SQL 2008 reporting services! So for SQL reporting 2005 you should visual studio 2005 instead!

Wouldn’t it be nice to design your own reports with nice looking bars or graphs with your company logo on it ?

Requirements before starting are:

• Visual Studio 2008 which is delivered with SQL 2008
• Authoring Console
• Feeling really artistic!

Before we start with opening the Visual Studio console you will first need to determine which counters you want to generate the report on.

The example will create a custom report for reporting on logicaldisk space. Since this is one of the most asked for and is missing from the default reports.

Counter(s) of interest

We are going to start with % Free Space.

This is the information we need to know to be able to collect the counters:

Counter : % Free Space

Object: LogicalDisk

The other ones are also of interest but to start with we need to know the object and counter to be able to setup a dataset to retrieve the information from the Datawarehouse.

The begin

Open visual studio and follow the steps below to start a new project in which we are going to create the report for disk space.

Go to new and start a new project. Select a Report Server Project and give it a Name.		After Creating the project we are going to create a report within this project. Navigate to Solution Explorer on the right side of the screen. (if not shown go to view and select Solution Explorer) Right Click on Reports and select Add New Item…
Select Report and give it a Friendly Name and click Add.		Now we have a report in our project. Next we are going to add an Item to the report. As stated in the design surface you can add one by using the toolbox on the right or you can right click and select insert.
When you right click and select insert you have the following options you can select. For our example we are going to select a Matrix.		When the Matrix is selected we need to define a dataset. This is going to be the query we are going to use to query the OperationsManagerDW. Give it a friendly name and select New next to Datasource.
The Datasource is the Shared datasource the Reporting server uses to connect to the OperationsManagerDW. The Name is “Data Warehouse Main” Unfortunately Visual studio 2008 doesn’t support datasource names with spaces… so we fill in “DataWarehouseMain” instead. We change this afterwards. Next connection string, simply edit browse your sql server for the OperationsManagerDW. The connection string to you data warehouse is used to test our reports directly from visual studio.		After Setting up the datasource we can add the query we are going to use. For ease click Query Designer… The Query Designer will open and you need to add your query in the box with SELECT FROM

Main Dataset Query

This is the query we are going to use. The SELECT and FROM part are for collecting the data from the Perf.vPerfDaily view which is the view for Daily data from the OperationsmanagerDW.

TIP! If you change vPerfDaily to vPerfHourly you will retrieve the data per hour!

The WHERE part is the part we are filtering the data on. Notice we are going to filter on date @Start_Date and @End_Date which will be the parameters we are going to define. This way we can tell from which date to which date we want to run the report on.

The data is filtered on LogicalDisk which will only retrieve Data from the Object Logicaldisk.

Last we are also filtering on @ServerName which will also be a Parameter we are going to declare for the servername we want to retrieve the data.

Lastly the data is ORDER BY date. This way the data is retrieved in a chronological way.

Well after pasting in the query you can run it by pressing “!”. It will ask for the parameters fill in a start date / end date and server name (FQDN). The query designer now shows the data and how it collected the data by joining the tables. How cool is that and you are starting to look like a developer now already with this cool code and models ;-) After looking over your result. You will notice we are retrieving more data then we are actually going to use in this example. No worries you will probably have enough inspiration after designing the report to use the other data as well! When you are done looking at your developer like screen you can click ok and save your dataset settings.		We are going to create 1 more dataset and after that declare the parameters. First the Dataset, let’s call it DataSet_Servers Use the following query: Select DISTINCT vManagedEntity.Path FROM Perf.vPerfDaily AS vPerf INNER JOIN vPerformanceRuleInstance ON vPerformanceRuleInstance.PerformanceRuleInstanceRowId = vPerf.PerformanceRuleInstanceRowId INNER JOIN vManagedEntity ON vPerf.ManagedEntityRowId = vManagedEntity.ManagedEntityRowId INNER JOIN vPerformanceRule ON vPerformanceRuleInstance.RuleRowId = vPerformanceRule.RuleRowId Where ObjectName = 'logicaldisk' This query retrieves all Server Objects to report on.
Declaring the parameters in the last thing before we can start with designing the report. These are our parameters: @Start_Date @End_Date @ServerName Go to parameters and go their properties.		At the parameters properties for @End_Date set the data type to Date/Time on the general tab. Next Default Values tab and select Specify values. This will set the default values for the parameters when opening the report. For Value click the Function button (fx) and now you are in the expression designer. Go to Common Functions\Date & Time\ and select today (double click). TIP!Also there are examples how to use this function.You can select each to find what it is used for! Function for End_Date which should be today =Today() Do the same for @Start_Date with function =DateAdd("d",-7,Today()) Finally the @Servers at the General Tab select Multiple values.Next open the Available values tab and select Get Values from Query. Fill the Dataset we created DataSet_Servers and fill in both Value field and Label field with Path. Simple use the selection to select the dataset and the values.
Now we can start with the design drag and drop the values from the report data pane onto the Matrix like on the right. You can now run a test report to check which results you get.		When you did test the report you noticed there where many counters and data. First we going to filter the results since we are interested in the % Free Space for this example. To filter the results we need to change the Column Group Countername. Open the group selecting the column and go to the group properties. Now navigate to Filters. Here we are going to filter our results. Click Add to add a filter and in expression select [CounterName] use the = sign as Operator and fill in % Free Space. Select Ok and run the report again to check the results. running the report will tell you how the data is presented and give you a quick solution to be able to visualize what you are actually doing. After Adding the filter we have narrowed the results down to only % Free Space. Except now the result is calculated all results together.
We need to change the results to 1 value which makes sense and probably the most up-to-date one. Select the cell and right click to open the FX Expression designer again. Almost every value in the report has a possibility to add expressions! This makes it very flexible! Now change the expression to the following: =Round(Last(Fields!AverageValue.Value)) The Round function is used to round the number so it’s easier to read instead of a PI like number ;-) The Last function is used to only retrieve the last value. Which is today because we ordered the query on Date we now this is the last number!		The result should now look something like the right picture. By adding the filter on the Countername we only see the counter % Free Space. And by adding the expression in the details part =Round(Last(Fields!AverageValue.Value)) we have the results shown. The screen is just from a test environment and the results are probably different but the basics should look the same. Computer \ Instances running on the computer and free space. The next steps will draw the graph in a more nicer way.

The Designing Part

The designing part really depends on your personal favor but I will show you how you can make your report look more sharp in a couple of minutes.

Let’s add some more to the report. Open the toolbox and drag and drop a Gauge onto the drawing area. Wow aren’t those meters looking sharp! Select the one you want to use and ok. The meter will be pasted into your design as a new object. Simply drag and drop your new meter object into your matrix and let it go in the cell with =Round(Last(Fields!AverageValue.Value)) .		First let’s go to report properties to check and maybe change the page settings of the report. The values given here you should pay close attention since when designing you should always make sure you stay within these boundaries. Especially when you want the report to be converted to like.pdf. When you go over the settings the graph will expand multiple pages which isn’t always nice looking. If you look at the default values here you will notice the width is 8.5 and margins both right and left 1 in this makes your drawing area 6.5 same goes for the height of your page which would be 9. Make sure your designing area is within 6.5 and 9! Simply click on your designing area and go to it’s properties if no properties are shown on the left of the screen click F4 to make them appear. Now change your design area to 6.5 by 9. This way you are always sure you are editing within the page limit! TIP! When you get blank pages it is most of the times because these boundaries are not set correctly!
The matrix should now look something similar. you can expand the matrix now from one point of your designing area to the other and make the gauge more visible. You can select the different objects which make up the gauge graph. The important one is the bar as shown. For both pointers add the expression =Round(Last(Fields!AverageValue.Value)) This will show the %free space collected value but now on the bar!! TIP! when designing the bar first make it look like you want and after this you can edit the size. This makes navigation easier ;-)		After playing around endlessly with colors and settings let’s continue we have a report to make  ;-) Let’s make a page header and footer to hold extra information like title company logo execution time etc. Right click on your design area to add a page header and footer.Here you can add a text box or image. In the text box you can also add expressions! Or drag and drop report properties to the text box. This way you can display logos, a title, creation date, etc.of the report. Save your report as .rdl file and open the SCOM Authoring Console.
Create a new Management pack and go to the reporting tap. Create a new report and give it a name. Next go to the Definition tab and select “Load content from file” Navigate to the .rdl file and import it.		Now you will see the xml version of your report in the authoring console. First remove the first line After removing this line we need to correct the datasource setting. Remove the connection settings Data Source=.;Initial Catalog=OperationsManagerDW         true       And add after the following Data Warehouse Main. TIP! Don’t forget to change the options tab the visible setting to True otherwise no report will be shown!!! Now you can save your report in the management pack. And after this you can import the management pack in your environment.

Result

This post described how to create a report for SQL 2008 reporting with visual studio 2008. Although it takes some time to get the graphical results you want you can now create a custom report.

This report is a simple example of how to create reports using visual studio 2008. I know there are more and possible better solutions and queries to get results but this is for another post ;-)

Have fun with expanding your reporting functionalities and drawing skills!

The Management Pack is a big improvement compared to the old version 1.4 which was basically a SNMP trap catcher.

The Management pack can be freely downloaded from the NetApp Now page. If you are a NetApp customer you should have access.

The official documentation reads

You can do the following with ApplianceWatch PRO 2.0:
• Use your existing storage management tools
• Use one interface to monitor all of your NetApp storage systems
• Simplify data management through integration with Microsoft System Center Operations Manager 2007
• View hardware and storage layout of all of your NetApp storage systems
• Isolate problems using alerts and Microsoft Health Explorer
• Monitor the performance of your NetApp storage controllers using different performance metrics
• Configure deduplication and autosize settings on NetApp flexible volumes
• Launch FilerView from the ApplianceWatch interface
• Connect to Data ONTAP DataFabric Manager servers from the ApplianceWatch interface
• Monitor storage availability and capacity of virtual machines running on Microsoft Hyper-V hosts
• Receive Performance and Resource Optimization (PRO) Tips with automatic remediation support

The following is NOT supported

• Traditional volumes are not supported.
• Data ONTAP GX and Data ONTAP 8.0 Cluster-Mode are not supported.

ApplianceWatch supports the following versions of Data ONTAP:
• 7.2.6 or later in the Data ONTAP 7.2 product family
• 7.3 or later in the Data ONTAP 7.3 product family
• 8.0 (7 mode) in the Data ONTAP 8.0 product family

Before we start

Needles to say since you are a good SCOM admin and are fully aware of the use of the installation manual before adding anything in your environment, right? ….. ;-)

Every documentation accompanied by a management pack is not for people who don’t know or for read when you have some spare time because SCOM is watching your environment for you ……but it is a absolute must before doing any importing or adjustments in the environment !

Okay after reading the documentation and knowing the limitations and actions to take we can continue.

Installation

After Downloading the management pack msi you can install (extract) the files.

The Actual management packs which you can import are:

• DataONTAP.mp for general monitoring your NetApp Appliances
• DataONTAP.PRO.mp if you are using System Center Virtual Machine Manager 2008 R2 and want to use PRO tips.

The other files are required to run tasks and to connect to the NetApp Appliances.

Before you can start with importing the management packs you need to setup SNMP on your NetApp Appliances and SCOM and make sure you add the NetApp Appliance as a Network device in your SCOM environment.

The procedure for doing this is described in the manual (a separate download from NetApp NOW). Also notice there may be hotfixes needed in your scom environment dependent on the version you use.

After making sure the NetApp Appliances are discovered and being monitored as network devices you can continue the installation extract (install) the MSI and import the DataONTAP.mp and optionally the DataONTAP.PRO.mp into you SCOM environment.

The next step is to fill in the login credentials for the NetApp Appliances you want to monitor.

You need to target your management server and start the task in the action pane DataONTAP:Manage Controller Credentials the following screen will appear:

Here you can fill in the credentials for the NetApp Appliances you want to use for monitoring and discovering the appliances. Simply select the NetApp Appliance and fill in your credentials. You also have the possibility to test the credentials.

The design of the management pack is not standard this makes it more difficult to understand and manage. By default you would want to store any additional credentials or accounts in the way SCOM is designed in your Run As configuration although the approach is similar.

Personally I think you want to use one approach for standardization and ease of usage which should be using Run As accounts but this is just my thoughts. 

Discovery

After setting the credentials to access the DataONTAP on the NetApp Appliances you need to turn on discovery.You need to turn on discovery on your management server because it is disabled by default. Well let’s take a look at the MP.

The Management pack has 1 discovery rule which is a group populator. This one is activated by default and it is not the one we are after. How does the discovery run?

Actually it is a rule which is doing the discovery, again different approach from most management packs. Let me explain….

The NetApp Appliancewatch management pack uses it’s own libraries, these are the dll files which are extracted into the installation directory of NetApp Appliance watch 2.0 they start with NetAppSCOM…..dll.

Besides the SCOM dll’s there are more which are being used to talk to DATA Ontap. This is the way the management pack is designed and it is not just a couple of MP’s.

This approach makes it hard to tell how the data is being collected or “read” the management pack because besides knowledge of Management Packs you should also have knowledge of the SCOM api and Data Ontap api and know how to read the dll’s files and understand the functions being used in the files!!

This makes it difficult for most people to understand what is happening under the hood. Although this is most of the times no problem, an admin who really wants to know what is being done at the NetApp Appliances can be hard to convince although the management pack is designed by NetApp.

Well back to turning on the discovery for NetApp Appliances. The rule which needs to be turned on by the use of an override is DataONTAP.Discovery.Rule and can be found under rules instead of Object discoveries.

Simply go to the authoring pane in your console select rules and search on “ontap” will reveal the rule we want to override. The override needs to be set at your Management Server, if you have more then one management server you can select which Management Server you want to use to run the discoveries on.

The default discovery interval is set at 24 hours or 86400. If you want, you can force a discovery by the use of a task DataONTAP: Run Discovery task which is targeted at the management server where you turned on the discovery rule.

After the discovery runs  your filers should look something  like this:

He, where did I see this diagram view before ;-)

When all components are discovered they will be monitored. The monitoring of the components is being done by the use of the newly created eventlog AppWatch. The monitors are triggered when certain event id’s occur and when the event occurs the mapping is being done by the UUID of the effected component.

Example:

This is an example of the events being created in the AppWatch log. These events are being picked up by the Monitor DataONTAP.Volume.UsedSpace.Monitor which monitors the AppWatch eventlog for events 4000 – 4001 – 4002 - 4003 The event ID itself will be monitored for the status of the monitor. In this example Event ID 4000 will tell everything is OK.

When an error event is detected the health status changes to warning or worse and an alert is raised. The UUID will tell which instance is having problems. When an event is detected everything is ok again the health status will change to healthy again.

Monitors

These are all monitors within the management pack. This to give an idea of what is being monitored by the management pack.

Also you have the possibility to receive snmp traps but these monitors need to be set by an override.

Pro Pack

The NetApp Appliance watch also includes a management pack to implement PRO tips for SCVMM 2008 R2. This management pack consists of a discovery rule to discover Hyper-v Hosts and virtual machines.

The management pack will send pro tips which can be manually changed or automatically by the MP.

To give an idea what exactly is discovered for a Hyper-V Virtual machine

The cool part is it actually discovers the relation between a Virtual Machine and the NetApp Storage to give a better view on the health status of your NetApp Storage and how it relates to the virtual machines! Nice Job!

The PRO tips which you may expect: 

Conclusion

Great to see more and more management packs from vendors which make the datacenter solutions for your entire datacenter monitoring possible! Especially combining System Center Products together.

Really a step forward compared to the old version NetApp Appliance Watch 1.4! For standard monitoring needs version 2.0 will deliver the basic availability of your NetApp Storage. 

A few pointers though from my personal perspective.

Still no performance collection for disk space which is one of the crucial parts which missing in the Management Pack. Don’t understand why they didn’t include it. Every SCOM project I run one of the first questions I get is does it monitor disk space? And if so can we report on it? Although the base management packs collect disk space performance you will certainly want to know how much storage is left on your storage solution. With today’s virtual environments adding space to a virtual machine is  just a couple of mouse clicks away..but adding space to your storage solution is another thing. Also it is always nice you can prepare your manager in an early stage he has to spend money and show him the graphs to convince him ;-)

Another nice feature would be to relate the luns used on Servers to the luns on the NetApp Appliance like they did in the pro pack. But without SCVMM this is not an option, although it is again one of the reasons to start thinking Hyper-V + SCVMM ;-)

The design of the management pack is not standard which makes it hard to understand what is happening under the hood. Therefore extending it or trouble shooting will be very hard or you will have to have developer like expertise. Maybe an extended expert documentation on the management pack would be really beneficial.

Although I did not have the opportunity to test this MP in a big environment and see it running together with Hyper-V and see it performing. I think this management pack can be a real addition to your monitoring solution with little extra configuration. Would be interested to hear from people who are using the mp about their findings!

And as with most annoying problems the solution was easy, well most solutions are easy if you know them ;-)

 First of all some back ground. There are like 20 domain controllers all running Windows 2003 and where working fine. All monitors running and after configuring everything According to the guide and knowledge available there where still  a couple of domain controllers which kept on complaining about the response times.

If you look at the default thresholds of these monitors:

Configuration
• Interval (sec) default 300.
• Number of Sample default 5. Number of times the threshold has to be exceeded before an alert is raised.
• Threshold Warning (sec) default 5. GC Search time threshold for warning.
• Threshold Error (sec) default 15. GC Search time threshold for error.

When I compared the value’s collected on the 3 effected domain controllers they where way off and when looking at the performance reports they where all around 200 - 300 average which means the response times where around 5 minutes!!!

Since I don’t like and want to adjust thresholds just to have a green view this asked for further investigation.

There are some tasks to check the response times like running the task AD General Response Task  when I ran this task manually it would give back a value way below what was shown in the performance graphs.

So this task was telling me there where no response problems!?!?! But the monitors showed all red with values above 200 and the performance graphs where also showing these values.

After checking the overall performance of the DC’s which resulted in nothing….. time to dive a little deeper into the Management Pack and it’s rules.

A quick review of the management pack in a unsealed version learned the response times where calculated by script based monitors. Which uses the script : AD_Op_Master_Response.vbs

Since you can always get a copy of the script on the agent (Default C:\Program Files\System Center Operations Manager 2007\Health Service State\Monitoring Host Temporary Files) and copy it to a temp location.

If you copied the script to a temporary location open the script and check which parameters the script uses. Now run the script in a command prompt (cscript AD_Op_Master_Response.vbs ) if the parameters are passed correctly you should be able to view the output in the command prompt.

When I tested the script and reviewed the output the values where all again below the thresholds…..  like 0,029

Hmm why where the values in the monitors and performance views so high compared to the values collected directly by the script?

After taking a break and discussing the outcome with a colleague of mine we both simultaneously said the same thing “Well good it be the regional setting aka the good old “comma-dot” mix up! “

A quick investigation of the effected domain controllers learned indeed the regional settings where different!

Domain controller with no latency problems	Domain controller With latency problems.

Changing these settings and on the advanced tab check the box “Apply all settings to the current user account and to the default user profile” and restarting the health service on the effected domain controllers resulted in green monitors with the correct values!

The problem was because these settings where active it caused the comma to jump a couple of places resulting in values of 290 instead of 0,029 when the value’s where converted after the script runs.

Lesson learned with this problem is double check your outcome and make sure data is converted, translated and read in the correct way! If you have strange high values always be sure to be able to explain why the values are high before changing a threshold!

This part will cover the parts which are still missing if we look at the properties of the classes. We are going to “discover” these properties which in return will make it possible to collect and monitor performance of the NetApp Filer.

Previous series:

SNMP Management Pack Example: NetApp Management Pack for SCOM 2007 R2 part 1

SNMP Management Pack Example: NetApp Management Pack for SCOM 2007 R2 part 2

SNMP Management Pack Example: NetApp Management Pack for SCOM 2007 R2 part 3

 As I stated in the first post I wanted to be able to monitor performance and not just alert on the performance but calculate health status based on the collected information. The big improvement instead of telling the device to send a trap if it is running out of disk space is that the health status will warn if there are problems and will return to healthy if resolved. Besides this it will also make it possible to do forecast and trend analyses using reporting and the performance data collected.

 Discovered Properties by Now

By using the previously  created datasources and discoveries we are now able to discover the following (picture right) classes and their properties.

With the lun discovery we didn’t go through the process of creating another datasource to discover the Index property. This is because we can reuse the same datasource as we used by discovering the Aggregate class property Index. Just create another discovery rule and use datasource NetApp.Management.Pack.Datasource.Discovery.ObjectIndex and target it to a NetApp Lun to discover the Lun Property Index!

There is again one of the benefits of creating your own datasources. You create one datasource and reuse it in your discoveries or monitors!

 

Below are the properties we are still missing to be able to discover these we are going to create a couple of more data sources.

We are going to use the same principles we used in our earlier discoveries and some more. These will give you a more detailed view on how the datasource principle is so beneficial if you are designing your own management packs.

Because of the design of the NetApp MIB we need to discover again an index value.

The reason for this is we are running our initial discovery at a different Index then we are going to use when collecting performance data.

  Again when browsing the MIB (picture left) there is a index table called dfTable as you can see in the description this is the one we need.

  This index contains all counters for the disk space performance collection. These are the ones we are interested in now how are we going to discover these and make sure we are collecting information for the effected file system?

Now when browsing the MIB deeper (picture right) under dfTable we notice an OID called dfFileSys This OID contains the name of the referenced file system which we are going to use!

We have now managed to create a couple of datasources which in their turn use different approaches for data collection. Combining the knowledge we should be able to come up with a solution!

The solution for collecting performance data contains 2 parts the first part is discovering the dfIndex OID. The second part is using this dfIndex OID to target the correct OID’s for collecting performance data.

First part dfIndex oid discovery which is the FSIndex property for aggregate and volume classes.

Let’s start with creating the datasource give it a name NetApp.Management.Pack.DataSource.Discovery.FSIndex

And use the following modules:

The datasource is based on previously created datasources it’s a combination.

The Scheduler and Mapper Modules are the same as in the datasource NetApp.Management.Pack.Datasource.Discovery.ObjectIndex.

The SNMPProbe is basically the same as NetApp.Management.Pack.Class.NetApp.Aggregate

 Now let’s start in a chronological way of how the datasource runs.

Scheduler

SNMPProbe

Notice the oid we are going to run a SNMP walk on is the dfFileSys which is the filesystem name.

 ValueFilter

The value filter is used to filter the results. Like we did with the datasource NetApp.Management.Pack.DataSource.Discovery.Lun

After making sure we have the value oid for the corresponding volume or aggregate the data is passed to the script otherwise the data is dropped.

ScriptProbe

After making sure the correct data is collected by use of the filter the OID is passed to the scriptprobe.Which will retrieve the last number of the OID which is the index number of the corresponding aggregate or volume.

Notice there are actually two components discovered FSIndex which is the indexnumber of the filesystem and SNIndex which is the index number of the snaphots hosted on the effected volume or aggregate.

When browsing the Mib and retrieve the data for dfFileSys which returns the following value’s:

Notice there are actually 2 filesystem value’s for each volume or aggregate. The reason is the first one is always the filesystem itself and the second one is always the snapshot usage.

By retrieving the first index FSIndex we are able tell the snapshot index as well!

If the first FSIndex is 5 then the snapshot index of this volume will be 6 and so on. Although I haven’t used the SNIndex yet in the management pack. I have plans to add the snapshots as well for performance monitoring.

Mapper

Remember the approach we took when designing the datasource NetApp.Management.Pack.Datasource.Discovery.ObjectIndex. This mapper is exactly the same so when designing the mapper use the same approach, save as default and edit afterwards in xml editor.

Finally for this datasource the configuration tab

Now we can use this datasource for discovering the FSIndex and SNIndex of Aggregates and volumes! Simply create a discovery rule target it at the aggregate or volume class and use the newly created datasource and fill in the missing pieces!(right picture)

Wrap up:

Now we have created the datasource and discovery let’s wrap up.

Let’s say we want to discover the FSindex of an aggregate named Aggregate1.

The discovery runs on aggregate1 which does a walk on the filesystemindex dfFileSys.

All collected values from the walk are passed through the filter which filters the value on aggregate1. Only the Value for Aggregate1 and it’s OID is returned.

Now the OID is passed to the script which trims it to only the index number and adds the index number in a property bag FSIndex. For SNIndex the FSIndex is increased by one.

Both values are now mapped to the Aggregate1 class properties FSIndex and SNIndex!

 Now we are able to discover the different classes and their main properties.

Although there are some more discoveries which we didn’t cover but with the information provided in the series you should be able to figure them out yourself ;-)

Performance Datasources

 Now with all the properties discovered we can start with the next part which are the performance counters. Although you are free to design your own I have narrowed it down to the main counters.

Basically we going to create the following performance counters

Creating these counters we are going to use 2 datasources

The first one NetApp.Management.Pack.DataSource.Performance.Percentage.Used is the easiest one it consists of tree modules a schedule a filter and a SNMPProbe.

The SnmpProbe sends a SNMP Get to the OID named dfPercentKBytesCapacity OR .1.3.6.1.4.1.789.1.5.4.1.6.$Config/Index$ where $config/Index$ is the FSindex property of the Aggregate or Volume Class.

That’s all to it!

In more detail we needed to discover the FSIndex for the aggregate classes and volume classes to be able to pass this index along with the correct OID to retrieve any value in the dfindextable which is shown on the right side. Notice all the OIDS shown in the index and their corresponding MIB name.

When using the FSIndex property we can send a snmp get to any OID listed in this index! And for percentage used this is dfPercentKBytesCapacity….. Although the name is not really clear ;-)

This approach we are also going to use in our next datasources.

Although creating the above datasource is optional because you could as well be using the one from the system.performance.library MP named System.Performance.SNMPPerformanceProvider!

There is one difference since I created the datasource myself I have the option to declare which $Config/ values I want to be able to set overrides on!

Default you can only enable or disable the one from System.Performance.SNMPPerformanceProvider.

And I wanted the most flexibility in the MP I created the datasource to be able to override the value for the interval.This way when sealing the MP I don’t have to worry about flexibility!

The next datasource we are going to design is NetApp.Management.Pack.Datasource.Performance.KBytesToGB which is the datasource for collecting the value and translating it to GB values since with today's disk space consumption it is easier to read GB instead of KBytes ;-)

If you want to use KBytes you are free to do so and don’t need this datasource in fact you can rely on the good old System.Performance.SNMPPerformanceProvider again but no overrides on the interval!

First I needed to find the values for UsedBytes this was a real journey! Why?

If you look they are right there dfKBytesUsed so just use the same approach as with the percentage used one would say. This works great as long as your values are beneath the 2 TB there is no problem with using this approach.

But when you go above the magic number of 2 TB (2147483647) something magical happens. The value of the OID changes to a negative value!

Instead of sending 2.1 TB it will show –2 TB (or something) and using the calculator in any possible way I know (and don’t know) I couldn’t come up with the correct value when collecting this OID.

This called for further investigation!

The reason for this magic number….. is not so magic when explained. These SNMP OID values are 32 bit Integer values and they can hold only SNMP Value up until 2147483647 (2TB) if it gets above this it will go into a negative value.

Well that explained the problem but didn’t resolve it. Looking further at the index the are actually 64 bits OID’s. Great!  These are the counters we are looking for.

There are also counters for High and Low, since we are using SNMPv2c which supports 64bits we have no need for them but just for the record. Back in the (old) days when SNMPv1 was/is used and they discovered the limitations of 32 bits they did some math to come up with a clever solution of writing the data to 2 OID values High and Low. You would need to do the math in a script to come up with the correct value like below:

if (Low >= 0) x = High * 2^32 + Low

if (Low < 0)  x = (High + 1) * 2^32 + Low

Back to the datasource we know the OIDs we are interested in and we have the FSIndex number so we are able to do our trick again.

Scheduler and Filter are familiar and take a look at the SNMPProbe.

Notice the OID values are actually two parameters combined in one. The reason for this is we can reuse the datasource in the following performance collection rules

Used – Total and Available!

ScriptProbe

A simple script to calculate from KBytes to GB and pass it to a property bag called SizeGB.

Now just create your custom Performance Collection rules and you are able to collect performance on your Aggregates and Volumes on disk usage. For the full information here are the basic steps:

Create a custom Rule and target it at the Aggregate or Volume Class create the following modules on the modules tab:

A Module for writing data to the Operations Database (PerfWriteOpDB) and one for writing data to the Reporting Database (PerfWriteRepDB)

 Monitor Type

Now we have created the performance collection rules I also wanted to create a Monitor type and explain how to. This way you can re-use this approach in your own management pack journeys ;-)

Before I start I have also created a Datasource to do a basic SNMPprobe called NetApp.Management.Pack.Datasource.Basic.SNMPProbe it is a basic datasource of a schedule, filter and SNMP probe module. For details see the Management pack itself.

Create A new Monitor Type and call it NetApp.Management.Pack.MonitorType.PercentageUsedSpace

States Tab

Make it a 3 state monitor since we want to monitor the percentage used till a warning state is raised and an error state is raised.

Create the 3 IDs for the three different states.

Member Modules Tab

Here we are telling which datasource to use to collect the value and what condition the data needs to have.

DS1

                              UtilizationOK                                                                                                 UtilizationLow                                                                                                              UtilizationHigh

Regular Tab - Configuration Schema - Overridable Parameters

Combine condition detection and status if the monitor.

Fill in the parameters to be use in the monitor.

Fill in the overridable parameters.

We have created our monitor type. And as it is the same with creating datasources you can change or configure them as you like.

Now just create a custom monitor target it at the class you want (aggregate or volume) and select this monitor type as it’s base and fill in the missing information.

Wrap up:

 We have covered the main areas in designing this Management Pack and I hope you found the information useful and will use the approaches explained in your own MP’s.

 As promised I will upload the Management Pack NetApp Management Pack for SCOM 2007 R2 to the site for everyone to use and have fun with it!

LINK: NetApp Management Pack for SCOM 2007 R2

Be aware though that using the MP is at your own risk because of the way SCOM and SNMP work the workload on your SCOM server can increase as well as on the filer you want to monitor.

I have tested the MP in different areas and as soon as the filers are getting bigger the more probes are being send and the heavier the workload is going to be on both sides! Although the MP is more tuned then described in this series always be careful when importing management packs in you environments especially when it is production. If available always use a test environment to be sure you understand the management pack and it’s impact! Closely monitor your environment before, during and after to be sure no problems arise.

Especially if you want to play around first try the NetApp Simulator like I did when developing this management pack.

I have created this Management Pack for fun and brains! To play around with SNMP and SCOM and to learn. I hope you do as well ;-)

The release of Kris his XSNMP management pack has brought a new approach for dividing the workload on SNMP monitoring! Great Job! Will look into it to it as soon as I have time to tune this MP based on the approach used. Also will be working on a SCOM 2007 SP1 version as soon as I have time probably within the next weeks.

Previous series:

SNMP Management Pack Example: NetApp Management Pack for SCOM 2007 R2 part 1

SNMP Management Pack Example: NetApp Management Pack for SCOM 2007 R2 part 2

First I will start by creating a new datasource for extracting the Index number.

This part is from Kristopher Bash his solution on creating SNMP management packs for network devices as I explained I used his approach as a baseline for this Management Pack.

Let’s take the last part of the Aggregate Discovery

Besides the OID and AggrName there are more properties which I will cover each in this series. As stated we start with the Index Property.

Why is this index number so handy?

Well the example for the Aggregate isn’t that great but let’s assume we want to discover more properties for the aggregate we add one for aggrRaidType.

How would we discover the aggrRaidType property for each aggregate?

We know we need to query OID .1.3.6.1.4.1.789.1.5.11.1.11 for the aggrRaidType but we also need to know the index number from the aggregate to query the correct one.

Above are the properties for each aggregate we discovered. We did a walk on the aggrName OID .1.3.6.1.4.1.789.1.5.11.1.2 for the discovery and the last number of the OID of each aggregate instance is the index number.

We are going to create a datasource for extracting (discovering) the Index number for each Aggregate class instance by using a script based discovery on the OID property of each aggregate.

Create a datasource named NetApp.Management.Pack.Datasource.Discovery.ObjectIndex Create 3 Modules:

The scheduler is the same as in our previous discoveries.

Script probe

The script uses 1 parameter $Config/OID$ which we are going to declare in the actual discovery. This parameter will be the OID property of the aggregate.

It passes the OID to the script which extracts the index number and put it in a property bag named “Index”

Mapper

This one is different then our previous one. The XML will look like below:

Be aware though that you cannot add these settings in the Authoring console. You will receive the error below:

A work around this issue is don’t change the default Mapper settings.

Fill in the Configuration SchemaTab:

Add Schema References:

Add Simple Configuration Schema:

Save your MP and open it in XML editor and change the values.

Now save it again and open it in the Authoring console again and add the following in your Simple Configuration Schema:

The reason for designing the mapper this way is we can use this datasource now universal in all of our discoveries for the NetApp Management Pack. I will show you the actual discovery to explain.

Create a new custom discovery rule NetApp.Management.Pack.Discovery.NetApp.Aggregate.Index target it at the NetApp Aggregate Class

As a Datasource use your newly created datasource NetApp.Management.Pack.Datasource.Discovery.ObjectIndex

We can now fill in the appropriate settings according to the Configuration Schema:

Simply add the appropriate values and notice we are actually not discovering anything except the index number!

All Settings are derived from already discovered components and because we designed the mapper in a universal way the datasource can remain the same only the parameters are changing.

The reason for first designing the datasource and discovery for the index number is we are going to use this one in our next discovery which is more challenging the NetApp Volume which is hosted by the NetApp Aggregate which is hosted by a NetApp Filer!

Discovery of NetApp Volume

Now this is where the fun starts. Because we want to discover a volume which is hosted by an Aggregate we need to design something which is more challenging.

When you browse over the MIB by SNMP and look for the volume property it looks pretty simple at first besides the aggrtable we used in our previous discovery there is also a volTable!

Just use the same approach as the aggregate one would say!

Well in fact not…….. let’s think about it for a while. We want the volume to be hosted by the aggregate and not by the filer. There is our first difference…… well just add another key property of the aggregate which hosts the volume in our discovery then… nope although this will get the volumes discovered correctly there is a big downside.

Since the discovery will be targeted at the Aggregate Class the discovery will run on each aggregate class and start discovering volumes. When testing this with one aggregate this looks ok. But when you add another aggregate (or more) you will notice the error. On each aggregate all volumes will be discovered! So each aggregate will host all volumes on the filer and this is simply not the case…..

Back to the drawing board then.

In our Volume discovery we need to actually discover the Aggregate as well. We will need to find a way to discover a NetApp volume and on which aggregate the volume is hosted in 1 discovery!

Well let’s browse some more the NetApp MIB and finding some value which will cover both the Volume as well as the aggregate. This way we can discover both in one discovery.

hmm remember our earlier Aggregate discovery, let’s go to the aggrtable again:

Now there is a very interesting OID when you look at it’s description “List of all the flexible volumes within this aggregate”

When you do a get on this OID you will receive a list of all volumes hosted by the aggregate!

Because we created a discovery to discover the index number property of an aggregate we are able to target this OID directly in 1 SNMP get! This is the first part of the solution.

Let’s start designing the Datasource for this one:

NetApp.Management.Pack.DataSource.Discovery.Volume

Scheduler

the same as previous datasources

SNMPProbe

Notice we are going to use a parameter index which we are going to fill with the aggregate class property named “index”.

Filter

 Script Discovery

Now for the script Discovery which is the second part of the solution!

First we need to pass the following parameters to the script:

$MPElement$

$Target/Id$

$Target/Property[Type='NetApp.Management.Pack.Class.NetApp.Aggregate']/AggrName$ <----------------------keyproperty of Aggregate class

$Target/Host/Property[Type='MicrosoftSystemCenterNetworkDeviceLibrary!Microsoft.SystemCenter.NetworkDevice']/IPAddress$ <----key property of a network device

"$Data/SnmpVarBinds/SnmpVarBind[1]/Value$" <------collected SNMP get Value of aggFlexvollist

We are now using our datasource to do a SNMP get and retrieve a value and after retrieving the value pass it together with more parameters to the scriptdiscovery. This is one of the real benefits of using datasources you can manipulate data to the wanted output.

Now for the configuration schema tab:

Now the datasource is finished for discovering volumes. Now you can again create a discovery rule target it at the aggregate class and fill in the correct parameters.

Wrap up:

The Datasource to discover a volume works as follows. We know the aggregate index number which is unique to each aggregate hosted on one filer.

Because we know the index number we can query the oid we are interested in expanded by the index number parameter of the aggregate which for aggregatename 1 would be .1.3.6.1.4.1.789.1.5.11.1.9.1

This SNMP get with OID number .1.3.6.1.4.1.789.1.5.11.1.9.1 will retrieve a value: a list of volumes divided by spaces belonging to aggregatename 1

This list of volumes residing on this aggregate (Aggregate 1) is now passed to a discovery script where the list is divided by spaces and for each volume a new volume class is created with it’s key property Volname!!

Because each aggregate queries the oid attached with it’s own index number we are sure we are collecting the correct volumes for the effected aggregate!

Discovery NetApp Lun

Okay now for the last one in the chain. Again our previous approaches are not sufficient for this one so I needed to take another approach to discover the NetApp Luns.

Although as stated before it is possible to discover the NetApp Luns if they are hosted directly by the NetApp filer it is more challenging and fun to let them be hosted by the actual part which is the volume! ;-)

First let’s take a look at the class we want to discover.

Remember as with our previous discoveries, discovering the key property of the class and it’s hosted classes are essential! Discovering other properties can be done in a second discovery.

Again we use our SNMP browsing tool to look for the appropriate value that will deliver us both the Volume key property as well as the Lun key Property

After browsing you will notice the lun table is beneath blocks instead of filesys as our previous OID’s.

Now when we do a SNMP walk on the Lunindex we notice a value called LunName. The description sounds interesting “The name of the lun. This is also the path to the LUN on the Filer.”.

When we do a get on this OID you will notice the path saying something similar to /vol/vol1/lun

This path or lunName value includes the hosted volume name!

Now let’s build the datasource NetApp.Management.Pack.DataSource.Discovery.Lun

Member Modules:

Scheduler

Same as previous datasources

SNMPProbe

The SNMP probe is the same as doing an aggregate discovery only we are now querying another OID.

Filter

Now this one is different and this is the part what makes the solution work correctly!

Notice the high lighted part! The filter is a regex expression which filters the Value collected.

let’s take the earlier example of /vol/vol1/lun1

The filter looks at the collected value and searches if the value contains substring /vol1/ if it does the value is passed to the mapper if not the data is discarded.

(The reason for adding “/” is if there are volumes named the same as the lun the filter will not correctly work)

Since we target the discovery at the netapp volume class for each volume the discovery will run but will only retrieve discovery data if the value is passed the filter!

Mapper

The mapper looks the same as the one we used in our previous discoveries.

As for the configuration schema:

Now the datasource is finished for discovering Luns. Now you can again create a discovery rule target it at the Volume class use the new datasource and fill in the correct parameters.

Wrap up:

The discovery of the netapp Luns works as follows we do a SNMP walk on the LunName OID .1.3.6.1.4.1.789.1.17.15.2.1.2 and retrieve all LunName Values.

We pass these values through the Filter and only the ones which have the correct volume name in the LunName value are passed to the mapper to be discovered!

Wrap Up Part 3 of the series

We are now able to discover all our classes. If we look at a diagram view for one filler the results should be something like below.

<----------------NetApp Filer

<---------------NetApp Aggregate

<---------------NetApp Volume

<---------------NetApp Luns

The next series part 4 will cover the next steps in the MP which are discovering and explaining other class properties and creating performance counters.

Also there will be a example MP in it which have all the components as explained in the series….

With this information you should be able to create the classes and relationships we are going to use in this MP. The result should be something similar to the screen shots below.

We have now setup the classes and relationships it is time to start designing the discoveries. I will tell step by step how to design the discovery for each class.

A handy tool at this time besides the authoring console is a SNMP walking tool this way you can send a SNMP GET or even a WALK to the SNMP device to query the data.

In part one there are linked tools.

During this tutorial I assume you have been able to collect a copy of the NetApp simulator tool or have NetApp filers running in your organization. Even without a NetApp Filer I think the tutorial is detailed enough to understand the steps and use these steps in your own SNMP management pack!!!

SNMP Setup

First you need to make sure SNMP is setup correctly on both the management server and the NetApp filer(s). Without the correct setup we are not going to discover anything ;-)

SNMP setup basically involves 2 parts the first part will be the Ops Mgr Server and the second part the device you want to monitor in our example the NetApp Filer. The second part of the procedure is the same for every SNMP device you want to monitor. Although the commands may differ the basics are the same.

1. Setup SNMP on your Ops Mgr Server.

Navigate to your services MMC (Run Command “Services.msc”) now find the SNMP Service, if not available install the SNMP Service.

Control Panel –> Add/Remove Programs –> Add/Remove Windows Components. Now select  Management and Monitoring tools click details and select  Simple Management Network Protocol (or SNMP).

After installation you will have two services SNMP Service and SNMP Trap Service. The configuration needs to be done at the SNMP Service. Now open SNMP Service and navigate to security tab. Default setting is to only receive SNMP from the local host so we need to add the IP Address of the Device we want to discover / Monitor with Ops Mgr. Adding the device by Host Name, IP Address or IPX Address will allow communication from the SNMP device to Ops Mgr Server.

1. If you have setup SNMP on the Management server you need to make sure SNMP is set on the NetApp side as well by adding the Ops Mgr server as a SNMP trap receiving server. Logon as root on your NetApp Simulator and run the command SNMP this should give you the current settings. SNMP ? will list you all command parameters. Below is a screen from adding the Ops Mgr server 192.168.112.12 and add a community name test.

Now your environment should consist of at least one NetApp Filer and your Ops Mgr environment with both the ability to send and receive SNMP traffic.

Now with this setup you should be able to install a SNMP tool on your management server. After the setup Import the NETWORK-APPLICANCE-MIB into your SNMP tool to be able to query the NetApp filer and be able to make up something of the OID’s and values. I used Ireasoning tool

When you have added the NETWORK-APPLICANCE-MIB  make sure you fill in the IP of your filer and the community name

Discovery

I will walk you through setting up the discoveries in a logical manner and in detail.

NetApp Filer

First we start with discovering the NetApp Filer which is the “easiest” one since we simply run the discovery on the class Microsoft.SystemCenter.NetworkDevice.

Datasource

Let’s start with developing the DataSource for this discovery:

Simply go to Type Library in your authoring console and create a new datasource; NetApp.Management.Pack.DataSource.Discovery.Filer

Now give it a logical name like “Datasource to discover NetApp Filers”

Next go to the Configuration Schema tab.

Here you can declare the parameters used to pass to the datasource.

The ones we are going to use are:

These will be the parameters that needed to be provided in the discovery rule in order for the datasource to run. All these parameters are referred to by $Config\

(We we are designing the discovery rule later on the reason will become more clear.)

Datasource part 1 Scheduler

 First the scheduler create a new Member module by Add.. and navigate to System.Discovery.Scheduler, give it a name like “Scheduler” leave the default settings and click OK.

Now most of the times when using the authoring console it is much easier to edit the XML. Therefore open the just created scheduler by selecting it and click Edit.

Now these are the default settings for the scheduler:

We are going replace these settings with our newly created Parameter form the Configuration Schema Tab $Config/Interval$ (remember when there are $ signs used Ops Mgr will interpret the string as a parameter)

  $Config/Interval$ <----------------------------This is the Interval Parameter declared in the Configuration Schema Tab.

The scheduler will tell how often the discovery should run by configuring a parameter we will be able to change this when using overrides. This will make the MP much more flexible in terms of running discoveries.\

Datasource part 2 Filter

 This filter is used to check if we have the correct device OID. For NetApp Devices the OID always starts with .1.3.6.1.4.1.789

Add a new module and now select System.Expressionfilter

The default values are

We are going to change the expression to a regex expression and filter the OID value on .1.3.6.1.4.1.789

In order to do this we need to change/ add the following:

$Config/SystemOID$ <-------------------------------------------------This is the SystemOID Parameter we declared in the configuration schema tab.

1.3.6.1.4.1.789 <--------------------------------------------------------The Enterprise OID for NetApp Appliances

 If A network device doesn’t contain this OID then it is not a NetApp device and therefore the datasource is stopped and the device is not discovered.

If it contains the OID we can go on with the next module the Probe itself.

 Datasource part 3 SNMP Probe

 Again follow the procedure to add a Module System.SnmpProbe and edit the module in xml.

The default values:

Now this is where we are going to declare which OID’s we want to discover form the device.

This is the moment we are going to use our SNMP browser to connect to the NetApp Filer and use net MIB to analyze which properties we want to discover of a NetApp Filer.

Above screenshots are the parts of the MIB we are going to use. (You are free to select any other property /OID you want)

First of all the first part should be familiar since these are the values you used in setting up SNMP on your NetApp Filer.

false <-------------------------------------------------------Needs to be false otherwise you are telling it to write instead of collect!

$Config/IP$ <----------------------------------------------------------------------------This is the IP address parameter declared in the Configuration Schema Tab.

$Config/CommunityString$ <------------------This is the Community name parameter we declared in the Configuration Schema Tab.

 Datasource part 4 Mapper

 Add a new module System.Discovery.FilteredClassSnapshotDataMapper. The default values for this mapper are:

Let’s change the default value’s to the value’s we are going to use:

The first part of the XML is the filter to be sure correct data is collected.

The Second part is the class we going to map the discovered properties to.

The Third part are all settings collected by the SNMP probe are mapped to the actual properties. (notice also the IPAddress is mapped since this is the key property of the Network Device.)

Rollup.

Now we have created our own datasource which we can use in the actual discovery rule. Creating a datasource is a very valuable way of authoring your own rules. It’s very flexible and you can manipulate the data the way you want. (Although there are some pointers regarding SNMP walk which we get into later)

The datasource contains of the following components

 Now we configure the actual discovery rule with ease.

Create a new discovery rule with a name like NetApp.Management.Pack.Discovery.NetApp.Filer and target the discovery at Microsoft.SystemCenter.NetworkDevice.

Discovered Classes will of course be NetApp Filer class.

Now the configuration Tab is the place where we are going to use the newly created datasource. Click Browse for a type and search for your datasource - NetApp.Management.Pack.DataSource.Discovery.Filer

When the correct datasource is selected this is the place where we find the parameters from the Configuration Schema tab of the datasource again!

 You can really simply fill in the correct settings by clearing the value and click the arrow as highlighted this way you can select the properties of the target we are using Microsoft.SystemCenter.NetworkDevice.

NetApp Aggregate

Now this one is getting more complicated. We are not going to the details again of creating the datasource. Instead we dive in at the part which is different to the previous one NetAppFiler. Add a datasource called NetApp.Management.Pack.Class.NetApp.Aggregate.

 Now again create the following modules:

Scheduler – System.Scheduler

Filter – System.ExpressionFilter

SnmpProbe – System.SnmpProbe

Mapper – System.Discovery.FilteredClassSnapshotDataMapper

Filter Settings: now we are filtering on OID 1.3.6.1.4.1.789.1.5.11.1.2.

SNMP Probe

Instead of just sending one “SNMP get” to do the discovery we are now going to use the “SNMP walk” feature specific to Ops. Mgr 2007 R2.

Again we are going to use the SNMP Browser tool to be able to tell which OID’s we are intrested in.

We are going to do a SNMP Walk on the OID .1.3.6.1.4.1.789.1.5.11.1.2. which translates into the aggrName.

By doing a walk we tell the SNMP probe to walk the aggrName OID index this means getting

 .1.3.6.1.4.1.789.1.5.11.1.2.1

.1.3.6.1.4.1.789.1.5.11.1.2.2

.1.3.6.1.4.1.789.1.5.11.1.2.3

and so on until the end of the index.

This will collect the OID’s and their values. (In this example we have three aggregates so three oid’s and their values are collected.

 In XML this will look like below:

Notice the settings added telling the SNMP probe to do a walk instead of just a get.

The Mapper Module looks the same as the discovery of the netapp Filer:

 Notice we also need to retrieve the IPAddress property of the NetworkDevice. This is required since the NetApp Aggregate is hosted by the filer. To be able to tell which filer hosts the aggregate we need to also retrieve the key property of the hosted class. If we leave out the key property of the hosted class in the discovery it won’t be able to tell where the discovered class instance is hosted and will not discover anything!

Also notice we declared a key property for the Aggregate which is AggrName, this property needs to be

After saving the datasource we now have a datasource to discover NetApp Aggregates! We only need to create a discovery rule and use our Aggregate discovery datasource in the discovery.

The procedure for creating the discovery rule is the same as the filer with the only difference we can now target the NetApp Filer. This means the discovery will only run on NetApp Filers.

Wrap Up:

We can now discover NetApp Filers and Aggregates!

We can discover a NetApp Filer and all of it’s properties and also NetApp Aggregates with only 2 properties which are hosted by NetApp Filers.

Because of the complexity of the discoveries for discovering NetApp Aggregate properties and NetApp Volumes and Luns I will cover these in part 3 of this series.

Which will follow probably next week.

The Reason

I have written this management pack to get the monitoring of your NetApp environment into System Center Operations Manager 2007 R2. I know there are other ways of monitoring your NetApp solution like NetApp Operations Manager and there is even a NetApp Management Pack from NetApp.

Although the NetApp MP looks promising at a first glance it is nothing more then basically 3 rules which catches SNMP traps based on severity “ Information, Warning, Error”. The only difference is that it uses the NetApp mibs to translate SNMP data into the alert description.

Because it only generates alerts it says nothing on the health status of your NetApp filers. Besides it has no health status and rollup there is also no performance collection. So with some basic snmp rules you would achieve the same which is by far not what I wanted!

Now because of these limitations I decided to start playing around with SNMP and the NetApp Simulator in my SCOM 2007 R2 test environment.

Although if you use NetApp Operations Manager along with SCOM it may give you a workable solution. But still you will have another console to use and I wanted everything in the good old Ops Mgr console ;-)

Therefore I decided to write a custom management pack for NetApp filers which uses SNMP. The Management Pack and idea is based on an example from Kristopher Bash for monitoring Cisco devices.

These series will cover the total solution of designing the MP it is in a very detailed level so even though you don’t have NetApp running these series will be beneficial in designing your management packs. This way I can maybe help other people with building their own.

When writing these series I noticed the announcement for xSNMP Management Pack Suite which looks really promising! Especially the performance part! Will look into this as soon as it is available!

Requirements 

The tools used when desiging this MP are the common tools:

SNMP with Ireasoning

XML Editor with Notepad++

Besides these tools I created a test environment with Ops Mgr 2007 R2 and installed the NetApp simulator. This simulator acts as if it is a real Netapp Filer appliance it can be download from the NetApp website (login required). The cool thing is after installation you can also query the device by SNMP in your test environment ideal if you want to do some serious testing!

Maybe needless to say but always do your testing in a test environment!
 

Approach

Before we start with SNMP tools, mibs and authoring console it is crucial to design your model, classes and their properties and relationships. Therefore for every management pack you want to create start with a design and name your goals.
 

• How does the solution work?
• What tells me the application is running smooth?
• What classes make up the application and how do they relate to each other?
• How can I collect data?
• Which performance data is relevant?
• What do you want to monitor and how should it reflect the health state?

There are more but if you can answer these questions you should have enough information to start with the design.

Designing the Management Pack

First I started with designing the different parts (classes, objects) which made up a filer. Please always start with a clear design, because adding parts later on can be a real challenge if your classes are hosted and their relationships!

Basically the main classes for a NetApp Appliance are Filer – Aggregate – Volume – Lun

 

 

Remember a Hosted class cannot exist without its host!

When you look at the design

An Aggregate cannot exist without a Filer ------------> Aggregate Hosted by a Filer
 

Voila we have now already created our Classes and relationships!

Design Discovery

Now we have named our classes we need to start designing what kind of properties (attributes) we want to discover.

First let’s start with the filer we want to discover. This is the easiest one since we are targeting the discovery on the Base Class Microsoft.SystemCenter.Networkdevice we don’t need to add any key properties.

The properties we want to discover for a filer are:
 

  
 

Since its base class is Microsoft.SystemCenter.Networkdevice it automatically inherits the properties of the base class which are:
 

  
 

This means a filer has all its own properties and the ones it inherits from the networkdevice class.

The screenshot shows all properties belonging to a Filer Class. 

Notice a Network device has a key property of IPAddress. A filer inherits the properties from a network device so we can use them in the rest of our management pack design!
Because the Filer its base class is a network device which is a non-abstract class the filer can't have a key property. Therefore we don't need to add one. 

 

The other classes have hosted relationships and therefore a key property is required. For a class to be hosted by another class you need to declare key properties. Why is this needed?

When you discover a volume you will also need to discover on which aggregate the volume is hosted and on which filer. A key property is what makes the difference between instances of a class.

Let’s take the earlier example again 

When you are discovering a volume (instance of a volume class like volume-2) the discovery will also need to know on which aggregate (instance of an aggregate class Aggregate-1) the volume (Volume-2)resides.

As you can see one filer can host multiple aggregates and each aggregate on its turn can host even more volumes. So how can you tell the difference of an Aggregate….well in our case by its name!

There is our key property for an Aggregate!

Now we know the unique property of an aggregate we need to discover the volume and it's key property which is again its name! The same goes for the Lun class as well.
Our discovery which we are designing in the next post will discover the volume name and aggregate name. This way volume-2  is discovered which is hosted by Aggregate-1.
We have the key properties and we can fill in the other properties as well. Our total design is now complete:

 

  
 

Notice the key properties and the other properties as well. The important one is the key property the others you are free to create yourself.

I skipped a little part about the properties but with the SNMP tool you are able to tell which properties are appropriate for you. Next Series will cover this together with the actual discoveries.
 

Although there are a few pointers:

• Always be aware of the fact Ops Mgr is not SCCM..... Discovering attributes should only be used when you are going to use them in the future. When your purpose is collecting inventory System Center Configuration Manager is your friend and does this far more efficient then Ops Mgr.
• Also don't use Attributes which frequently change like free disk space...this also causes pressure on the discoveries.

The Next post Part 2 will cover authoring the discoveries.

During testing this solution only works on a 2007 R2 agent. I have tested this with both versions of the agent in both a SP1 and R2 environment and it only worked on the R2 Agent. 

Result form the R2 Agent:

Result from the SP1 Agent:

 

 

Full Alert Description for Consecutive Samples Threshold Monitor Type: