This goes against Micrsoft’s own best practice of not storing overrides and objects in the Default MP, since as we all know when you use the Disable feature it automatically stores these “overrides” in the Default MP. The best way to do what these instructions are saying is to set a standard override For all objects of type: and set the Enabled param to disabled.

Another example of bad MP documentation

But now that we know we can fix that with a simple override right… Right.

Seems like a strange thing to happen, but it does over and over again.

We’re currently working with a customer on monitoring their website. The monitoring is a mix of simple URL monitors as well as more complex end to end transactions. The complex transactions are using a domain account to log on to the sites and or web services. Seems simple enough, basic Operations Manager functionality. We are then going to use these transactions as part of 1 (or more) Distributed Applications (DA). In doing so this requires that all MPs related to these transactions be sealed. However what we are finding is that something in the export/import is breaking the authentication model of our transaction.

We created a URL synthetic transaction, used NTLM and associated it with a run as profile:

We then needed to seal this MP to use the transaction in a DA. We exported it, sealed it and reimported it with the following results:

In looking at the XML we see the user account is still associated with the transaction:

To test where the problem lies we deleted the sealed MP, and reimported the XML version and still got no User Account populated in the Proxy section of the Web App. I also tested this in three different environments with the same results. The main problem here is since the MP has been sealed we are unable to change the User Account field to add our Run-As profile, which means the transactions won’t run, which means we have no DA.

So unless I’m missing something it would appear that something in the Export/Import process that is breaking the Authentication/Proxy portion of the Web applications. We are currently investigating these problems and I’ll updated you when we find the problem. 

He stopped the System Center Management Service – the CPU dropped. Obviously this gets us thinking. I ran the %Processor Time report for the quarter and was able to pin point the day and time the CPU jumped from 20% to 60%. That day just so happen to be the day and hour we imported the ADMP.  Each of the drops in processor are periods in which the servers were rebooted, or we stopped the OpsMgr Service.

At this point we break out PROCMON and start looking for the problem scripts. After a quick capture and filter for CScript.exe we discovered that the ADMP was the culprit. I set an override on the Server 2000 DC Role and the proc dropped to a normal 10%-20% (as seen from the highlighted sections below).

At this point I started disabling rules and monitors that had the highest number of entries in our capture… DCLocator (had over 3600 entries), AD Lost Counter (had over 2200 entries), etc. I had worked my way through a majority of the monitors with no luck. Now I could have continued down this path to discover the actual rule or monitor that was the culprit; however the customer decided they wanted the ADMP removed from the Server 2000 DCs as they will be replaced with 2008 DC with in the next 2 months so the end result was to remove the Server 2000 AD Monitoring MP.

There are still Servers out there running Server 2000, and with these boxes we need to be careful that our monitoring solution isn’t causing more problems than it’s helping. just something to keep in mind when dealing with older technology.

The short term answer to this question is to go in to the Monitoring Pane, Web Application State View and put the Web App in to Maintenance Mode. In order to make sure the transaction itself has stopped we watched the performance counters on the Web App for Total Transaction Time. This particular counter went from .01 to 0 and stayed at 0 for the duration of Maintenance Mode. Using Maintenance Mode of the long term may not be the best solution, but it works for the short term. It would be nice to have a Disable Web Application check box in the vNext SCOM.

Say you you have an application that writes to a SQL table every few minutes, and if the application isn’t working the table value would equal 0. I would like to be able to use the wizard, enter the query as well as the expected result; alert me if the value is 0 if it’s greater than 0 it’s healthy. This still isn’t doable without creating a custom script.

I’m still very happy with the ability to query DBs to see if they are responding and to tell how long the query took to run, but I think this should be taken one step further with the ability to look for expected results.

While most of the work Microsoft has done on the URL monitoring components of Operations Manager have been on scalability the majority of URL Web application monitoring remain the same. You create the transaction using the same Add Monitoring Wizard in the Authoring pane. One of the pieces I truly don’t like about the wizard is you have to enter a URL and complete the wizard before you have the ability to Capture a Web Session (this is done in the advanced web application authoring screen). Once you have completed the capture session you have the ability to assign it to different watchers, choose what performance counters you want (you have to pay special attention here as by default very few counters are selected), choose how often it runs and customize any special alerts you want (like content matches, SOAP calls etc).

The next thing we have found in our testing is some of the performance views aren’t what they appear to be. In our case we have a web application with 15 individual requests in it (using the capture session for our synthetic transaction). We want to capture as much data about these requests as we can so we have selected all the performance counters for the transaction. After letting the transaction run for a bit we visited the performance views we created to see what kind of detail we have been collecting. The first thing we notice is how these counters “scale”. When using perfmon and changing the scale the graph changes to allow better focus on the data collected, however when we changed the scale in Operations Manager we find that not only did the graph change in some instances but the values on the graphs change. This struck us as unusual so we did some digging and really started playing with the numbers and as it turns out this number appears to be a percentage.

Our threshold of the total transaction is 20 seconds. One of the requests appear to only be taking .01 seconds. When we chance the scale (this is done by right clicking on the counter object in the lower details pane) not only does the graph change, but the values that have been collected change, leading us to the conclusion that what we are seeing is a percentage of the total transaction time. While this is valuable information, what we are wanting to see is how long this particular web request is taking.

We’re still playing with the numbers and counters, with any luck we will be able to put together a guide for our customer on how to read these charts.

Interested in Operations Manager, Config Manager, ACS, or MPDev training offerings, see http://www.infrontconsulting.com/

To start open the Console and browse to the Administrator pane. Right Click on Management Packs and select Import Management Packs.

On the Management Pack Import wizard window select Add, and choose Add from catalog…

The Add from catalog window appears and has a few options to aid in your search.

The first is the Find field. This allows you to filter through all the management packs.

The View: field allows you to filter even further. There are options to look for only the newest management packs, older management packs or even updates to management packs that may already be installed.

The updates option connects to the catalog and does a compare to the management packs that are installed and allows you to choose only updated management packs.

Once you find the management packs you are looking for, highlight the parent service or individual management pack you want (perhaps you are only looking for the Server 2008 OS management pack) click Add. If there is a prerequisite management pack (like a library MP) it will be automatically added to the download section. This assures all pre-reqs are met.

When you click ok you are taken back to the main management pack import screen. When you see all the management packs you are going to install (with the green check marks) click import.

What you will see is the status column change from blank, to Downloading, then Queued, and lastly Importing.

The Download Management Pack option allows you to use Operations Manager R2 to connect to the online catalog and pull down management packs you are interested in.

This wizard works similarly to the Import Wizard, the difference is instead of importing the management packs in to Operations Manager there is an option to choose a folder where the MP will be stored.

When you click add you are presented with the same options as before. Select your MPs and click OK. Once you have selected the management packs of choice, click Download.

As you see the wizard will download the management packs to the chosen destination. From that point you can select to import them manually, run them through MPViewer to see if they meet your requirements or examine the XML (if you are so inclined).

]]> Tue, 02 Jun 2009 01:54:00 GMT http://www.systemcentercentral.com/BlogDetails/tabid/143/indexid/19256/Default.aspx I have recently had the ‘privilege’ of setting up a R2 Gateway on Server 2008. Here are some caveats that you should watch for:

• Certificates in Server2008 have changed and as such make sure you are working with the correct cert types.
• The Cert Generation tool didn’t work for us; perhaps it was an environment specific thing, but it didn’t nonetheless and we had to resort to the old fashion way of manually requesting, exporting and importing certs. The tool didn’t add the Friendly name, the paths were incorrect etc. The old fashion way of requesting, exporting and importing worked just fine.
• The MOMCertImport.exe has to be run as an Administrator. By this I mean you have to right click on the tool and Run-As Administrator or you will get permission errors. The funny thing about the errors is you won’t see them. When you run the MOMCertImport GUI it spawns a separate cmd window that will close when the process is finished running, not allowing you to see that it failed with Access Denied. Nice error handling. I found a post on a news group talking about the problems with running this tool on server 2008. Basically again the end result was going back and running the tool in it’s original form, from a command window (an ADMINISTRATOR command window) and typing MOMCertImport.exe certname.pfx, entering the cert password. Here are the steps (not complete, but basic steps):

These steps are assuming you have correctly imported the Root CA certs, and submitted and installed the correct cert as laid out by the OpsMgr Guide.

1. Once you have a certificate imported into the certificate store:
   1. From the MMC “Certificates (Local Computer)” snap-in, locate the certificate (server FQDN name) in the Personal-Certificates folder, right click it, and select “Export”.
   2. When prompted to “Export Private Key”, select “Yes, export the private key”.
   3. Under “Export File Format”, select “Personal Information Exchange – PKCS #12 (.pfx)” and sub-option “Export all extended properties”.
   4. Take note of the location and file name of the pfx file you saved and password if you entered one.
2. From an Administrator mode command prompt, enter:
   1. MOMCertimport.exe filename.pfx (where filename.pfx is the exported certificate from above)
   2. - enter the password you entered when exporting the pfx file.

Once this is complete, the Gateway as been approved and the Gateway software has been installed you should see the Gateway in the console. This may take some time, but watch the Operations Manager event logs on both the Gateway and the Management Server for the following Events

Log Name:      Operations Manager
Source:        OpsMgr Connector
Date:          5/28/2009 8:51:16 AM
Event ID:      21023
Task Category: None
Level:         Information
Keywords:      Classic
User:          N/A
Computer:      COMPUTERNAME.FQDN
Description:
OpsMgr has no configuration for management group XXXX and is requesting new configuration from the Configuration Service.
 

Log Name:      Operations Manager
Source:        OpsMgr Connector
Date:          5/28/2009 9:28:36 AM
Event ID:      21019
Task Category: None
Level:         Information
Keywords:      Classic
User:          N/A
Computer:     COMPUTERNAME.FQDN
Description:
OpsMgr has returned to communicating with it's primary host MGTSERVER.FQDN

Another way to watch for this updating is to open the console and in the monitoring pane, open the Operations Manager Folder, then Health Service Configuration and click on the Config Update Events [Management Servers]. This view will show you all the pertinent events you will need to watch for.

Interested in Operations Manager, Config Manager, ACS, or MPDev training offerings, see http://www.infrontconsulting.com/

Interested in Operations Manager, Config Manager, ACS, or MPDev training offerings, see http://www.infrontconsulting.com/

Interested in Operations Manager, Config Manager, ACS, or MPDev training offerings, see http://www.infrontconsulting.com/

I clicked on it and walked through the wizard that allowed me to put my report in to a management pack.

Once I completed the wizard (and waited a few minutes) a new Folder was created in my Report menu, and my report was put in that folder.

I was curious about the XML output so I exported the MP and took a quick glance sure enough there it is.

Interested in Operations Manager, Config Manager, ACS, or MPDev training offerings, see http://www.infrontconsulting.com/

One thing that seems odd is on one of the screens you will see a check box that says “Second Node of a Windows Server Cluster”. This to me would indicate that it needs to be checked if you are installing this on the second cluster node, meaning we need to run the setup on both nodes?

The thing that strikes me as odd is when you are installing the Operations Manager DB you don’t install anything on the second cluster node, no executables or bits get installed on the SQL server itself. Why the need for check box? It can be a bit misleading, unless you’ve read the deployment guide, which I assume everyone has (at least twice… right).

We were able to use SetupOM.exe for the DW as well. While installing the DW we weren’t presented with this same option. There was no check box.

Also an interesting tidbit when creating the DW, setup forces a 50% Log file to DB ratio (DB = 500 Gigs, LogFile = 250 Gigs). Most DBAs I’ve worked with in the past like to see a 20% ratio, however there is no way to alter that in the setup. Just something to note for your planning.

Interested in Operations Manager, Config Manager, ACS, or MPDev training offerings, see http://www.infrontconsulting.com/

We checked all the permissions to make sure the OpsMgr container didn’t have some strange GP or security applied. We deleted and recreated the Computer object (within the OpsMgr Container), as well as ran a few more tests. What we had to do to get this working was to put the object back in the default Computers Container. Once the resources were successfully created and came online, we were able to move the Computer Object back to the OpsMgr container without further issue. I know it’s not really an OpsMgr R2 issue, but still very strange behavior. I’ve built a few 2008 Clusters and haven’t seen this issue before.

Interested in Operations Manager, Config Manager, ACS, or MPDev training offerings, see http://www.infrontconsulting.com/

The biggest thing is (in my opinion) Live Migration (or VMotion to you VMWare folks). Live Migration allows you migrate from one host to another with “no downtime”. In order for this to work the hosts must be a part of a Failover Cluster, the procs must be the same speed type and step. As part of a cluster they must have shared storage (CSV or Clustered Shared Volumes are highly recommended) to keep the “no downtime”. A dedicated network will used to handle the migrations. A host can only have ONE Live Migration happening at a time (migrations on or off). Retries happen in the background and will stop after 15 minutes of being unsuccessful.

VMM Maintenance Mode will kick off Live Migrations automatically moving VMs to the highest rated machines in the cluster, then putting the hosts in Maintenance Mode to stop host alerting.

Clustered Shared Volumes are now supported. This enables multiple nodes in a cluster to access a singled shared LUN. The VMs on these host can also access these LUNs. VMM detects who owns the CSV volume and will deploy the VM directly to that owning host. This is different than in the past as VMM would deploy to a host, then that  host would then detect the owner of the LUN and move the VM to the owner. This new direct deployment should save time and resources.

Storage Migration, will migrate the storage to a new location. It works with Quick Migration in Hyper-V and Vmotion in VMWare.

Lastly… Hot Add storage. VMM R2 allows administrators to add storage on the fly with out shutting down the VM first. I know as a former server engineer this feature will make a lot of people happy.

Interested in our training offerings, see http://www.infrontconsulting.com/

Diagnostics and Recoveries: Using the new R2 Authoring Console you can create recoveries to run based on diagnostic data, then once that recovery has run re-run the diagnostic to make sure the problem is truly fixed.

SLA Dashboard 2.0: The new SLA Dashboard allows for near real time SLA monitoring of both Mean Time to Repair (MTtR) and Mean Time Between Failure (MTBF). It supports Sharepoint 3.0 and includes Site Templates and Webparts to allow admins to quickly and efficiently create new sites. The console can be installed any machine that has OpsMgr access like a laptop or shared desktop.

Now some things I’m not sure I am 100% on board with but may be worth mentioning. During one of the sessions about performance and scaling, it was mentioned that the more consoles you plan to run the less agents you can support; something to the tune of 50 consoles = 6000 agents, 20-25 consoles = 10,000 agents. I would imagine this is concurrent sessions. Most of the organizations I’ve seen don’t run 50 concurrent sessions. They didn’t mention anything about SDK calls from connectors or third party applications that use the SDK, this would have to have some kind of impact. It was also mentioned that limiting Diagram views would save resources as would using scoped consoles more effectively and more often (the less database calls being made from the consoles the better).

Microsoft has also said they have changed the way they are collecting statics to help alleviate some of the performance problems that were being experienced in SCOM SP1 web applications. Knowing first hand that once you get upwards of 1500 Web applications you will have to scale your watchers out to 12+, and you may still have significant performance problems.

UPDATE - The fixes Microsoft has put into SCOM to help fix the URL monitoring performance issues are a reduced number of base performance metrics being collected, and the use of a single set of class and monitoring workflows per web applicaiton.