Agent Installation Issue
Home  » Forums  » Agent Installation Issue

Agent Installation Issue
Posted: Mon, Jun 08, 2009 4:25 PM :: Rank: 1
Author
Points: 4540
Level: System Center Specialist

I have two similar servers - Windows 2003 R2 with no application installed.
I installed an agent from the console to both these servers, one after the other.
For 123 server it installed fine and is green. But for xyz server it keeps on displaying the eventid 21023 - OpsMgr has no configuration for management group XXXXX and is requesting new configuration from the Configuration Service.
Its gets no other information and every 5 mins displas the same event.

I tried to repair - didn't work
Tried unistall  and then reinstall - didn't work
Tried to uninstall, run momclean.exe and then reinstall - didn't work.
Uninstalled, and reinstalled using different management server - didn't work
Unistalled, cleaned, ran a manual install from the local server and approved - didn't work.

I am not sure what I am missing, but this one server is too stubborn and would only display as "Not Monitored".  Not sure how I can get the server to be green.

There are no issues with the OS or anyother thing, as both the servers are identical hardware and OS. So 123 is fine and xyz is stuck.

Any other ideas?

Reply  
Report Abuse
Re: Agent Installation Issue
Posted: Mon, Jun 08, 2009 4:59 PM :: Rank: 0
Author
Points: 250
Level: System Center Hero
Delete the system from SCOM everywhere you can find it. Under Administration...Settings...Server Security make sure "Review new manual agent..." is selected. Manually re-install the agent on xyz In a few min, xyz should show up in the "Pending Management" folder under Device Administration; right click it, and approve the installation. This presumes that xyz can communicate with its management server on TCP 5723 (I'd check this first!)
Reply  
Report Abuse
Re: Agent Installation Issue
Posted: Mon, Jun 08, 2009 5:08 PM :: Rank: -4
Author
Points: 4540
Level: System Center Specialist
Thanks amduomsdca. I already tried that and installed the agent manually. The process worked fine. When I run the netstat, I can see that the xyz has established a connection with the management server on 5723. But as I said, all I get in the eventlogs is event id 21023.
Reply  
Report Abuse
Re: Agent Installation Issue
Posted: Mon, Jun 08, 2009 5:10 PM :: Rank: 0
Author
Points: 19054
Level: System Center Expert

When you have a manually agent in OpsMgr 2007 that cannot retrieve it's initial configuration successfully, there are a few things to check. The issue would seem to have something to do with either approval of the manual agent installation or mutual authentication.

Check Server Security settings OpsMgr Global Settings  -  Note that manual installations are rejected by default. Sounds like you have that covered already

Network Connnectivity - be sure TCP 5723 is available in both directions between MS and agent. Seems unlikely here as well

At this point, you should check the OpsMgr Event Log on the mgmt server it reports to. Filter down to warning and critical events and see what is displayed there. There is a wealth of information in this log on the MS / RMS just as there is on the agent.

If you can have a look there and post any events of interest, that would be great.

Reply  
Report Abuse
Re: Agent Installation Issue
Posted: Mon, Jun 08, 2009 5:17 PM :: Rank: 2
Author
Points: 4540
Level: System Center Specialist
On the management server logs I do see the eventID 20020 for the xyz server. Description - The health server running on host xyz.domain.com and service management group XXXXXX with id{ } is available.. But strangely nothing else is showing up on the management server. and the xyz server ops logs just show the event id 21023 every 5 mins..
Reply  
Report Abuse
Re: Agent Installation Issue
Posted: Mon, Jun 08, 2009 5:34 PM :: Rank: 1
Author
Points: 250
Level: System Center Hero
Ok, now it's time to check for the stupid stuff... In the registry on server xyz, make sure that there's a key that's SPELLED CORRECTLY for your SCOM MG: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft Operations Manager\3.0\Agent Management Groups\mymanagementgroup (when I do manual installs, I goof this sometimes, so this sort of thing will happen, even if you spell the server name right!)
Reply  
Report Abuse
Re: Agent Installation Issue
Posted: Tue, Jun 09, 2009 3:06 AM :: Rank: 0
Author
Points: 20153
Level: System Center Expert

If the agent action account does not have enough privileges some of the properties of the server will show up as being not monitored. Check the OpsMgr agent event log for event 1201. The agent health service normally will log that event for each management pack it downloads. Alternatively you can also go to the Heath State folder under the %Program files%\Microsoft System Center Operations Manager 2007\ (there should be a management pack folder) you can check to see if the agent downloaded the various MPs. If there are no MP’s in the folder it is another sign that agents are not getting configuration from the Management Server.

 Check your SPN’s (http://blogs.technet.com/smsandmom/archive/2008/03/13/opsmgr-2007-agents-stuck-in-pending-management-with-event-id-21016.aspx )

What Version of Ops Mgr are you using?

Is this a multi homed server? This can create this kind of behavior

Check that the server has Microsoft Core XML Services (MSXML) 6.0 which needs Windows Installer 3.1

Check the sizes of the Health DB on the box C:\Program Files\System Center Operations Manager 2007> Health Service State>Health Service Store inside there is the HealthServiceStore.edb compare the size to the other box.

Has the box been renamed?

Open the Cert’s in the MMC check that there is only one cert in the Operations Manager, if not delete all certs in the Operations Manager cert then stop the health Service (net stop healthservice) and start (net start healthservice) refresh the MMC is there one new cert (remember this is always Not Trusted.

Reply  
Report Abuse
Re: Agent Installation Issue
Posted: Tue, Jun 09, 2009 9:25 AM :: Rank: 0
Author
Points: 4540
Level: System Center Specialist
This is the most weird thing I have ever seen. Thanks guys for all your help, but still its no resolution. Andy - Yes, the Management group is correct in the registry. Simon- Here are the answers Agent Action account has the proper perimssions. I installed the agent on two similarly built servers. same OS, hardware, permissions. One servers is fine, while xyz is not taking any updates. This is not a rebuild servers. A out of the box, new server. Have checked the SPN's and no issues on that end too. Using SCOM SP1. The server is not multi-homed. msxml is installed on the server. with sp2 I delete the certs as soon as I uninstall the agents.. so there is only one cert in there which is red x I unistalled the agent yesterday. and today morning tried to install it.. thinking it would have been cleaned from db. But again after installation.. the ops mgr event log is stuck at event id 21023 - OpsMgr has no configuration for management group XXXXX and is requesting new configuration from the Configuration Service. have no other option then to call MS and open a case with them..
Reply  
Report Abuse
Re: Agent Installation Issue
Posted: Tue, Jun 09, 2009 10:25 AM :: Rank: 0
Author
Points: 2580
Level: System Center Specialist

Sameer,

 

Just wondering , can you try to install newer version agent? can be RC or RTM ? not sure if its compatible or not but  just wondering what will happen.

 

Thanks.

Reply  
Report Abuse
Re: Agent Installation Issue
Posted: Tue, Jun 09, 2009 10:54 AM :: Rank: 0
Author
Points: 20153
Level: System Center Expert
Sameer, it's time for a sniffer! Would you mind sending me the Event Logs, Application, System, Operations Manager and Security one if you are OK with to simon.skinner@secquorum.com in a zip file Regards
Reply  
Report Abuse
Re: Agent Installation Issue
Posted: Tue, Jun 09, 2009 1:22 PM :: Rank: 0
Author
Points: 4540
Level: System Center Specialist

well, after enabling the kerberos logging, I see the Event id 3 on the system logs.. Bthe server name its showing in the event is a domain controller. To my surprise, I tried to install another agent on a new server, and it again stuck with the same event..

Well, installed the agents on yet another server and the same issue. From what looks like our Europe server is also facing the same issue..

Reply  
Report Abuse
Re: Agent Installation Issue
Posted: Tue, Jun 09, 2009 5:21 PM :: Rank: 0
Author
Points: 20153
Level: System Center Expert
Sameer, could you make that a little clearer? But what I see (think i see) is that the SPN's are not right, but can you make the thread a little clearer? (I know you have already said there are fine but could you check again?)
Reply  
Report Abuse
Re: Agent Installation Issue
Posted: Wed, Jun 10, 2009 12:00 AM :: Rank: 0
Author
Points: 2967
Level: System Center Specialist

Have checked the SPN's and no issues on that end too.

Just to rule out SPNs, have you ever changed the SDK, Config, or HealthService service account logins on your RMS or secondary management servers?  (e.g. Changed them from a domain account to the Local System account or vice versa?)

If you have ever done that, you can have two accounts in your domain with the same spn.  Looking up the SPNs for you current account is not quite thorough enough.  If you happen to have the same spn assigned to 2 accounts, you will need to delete the spn entries from the account you are no longer using.  This can get especially tricky when clustering your RMS.

So event 21023 is raised (normally) the first time the Health Service runs and the agent has yet to download any management packs.  I believe if you clear the agent cache, you would get the same event.  If your "approved" agent cannot download its management packs, then I would suspect either a network issue (dns, wins, firewall, etc) or SPNs.  If you have a secondary management server being used in your topology, your agent might have connectivity to the mgmt server's HealthService, but not your  RMS's Config Service.

 

 

Reply  
Report Abuse
Re: Agent Installation Issue
Posted: Wed, Jun 10, 2009 6:03 AM :: Rank: 0
Author
Points: 20153
Level: System Center Expert

Sam, thanks for the logs,

I have these questions,

  • Is port 5723 open across the network, as the VPN's may be locked down?
  • Can you confirm how many client do work and if any are they all in the same subnet?
  • Is there any Gateway servers anywhere?
  • Can DNS resolve the RMS/Management servers?

 Simon

Reply  
Report Abuse
Re: Agent Installation Issue
Posted: Wed, Jun 10, 2009 9:48 AM :: Rank: 0
Author
Points: 4540
Level: System Center Specialist
Ok.. I have opened a case with MS on this issue. So far nothing has been found, but would update you guys as to what would be the resolution to this..
Reply  
Report Abuse
Re: Agent Installation Issue
Posted: Wed, Jun 10, 2009 10:10 AM :: Rank: 1
Author
Points: 20153
Level: System Center Expert
Sam, let us know the answer I think we are all keen to know
Reply  
Report Abuse
Re: Agent Installation Issue
Posted: Thu, Jun 11, 2009 11:21 AM :: Rank: 5
Author
Points: 4540
Level: System Center Specialist
Finally the issue got resolved. We checked on the RMS server to see if it was getting the new MP's.. but the rms itself was not getting any data from the DB. So we checked the DB for any rules that were broken - none we checked to see if there were any lingering objects in the DB - and yes we did find one. we checked what was the server and that id was showing two servers - strange. we took backup of db and deleted that entry.. And BINGO.. everything started working fine.. all new agents were healthy again...:)
Reply  
Report Abuse
Re: Agent Installation Issue
Posted: Tue, Sep 29, 2009 7:21 PM :: Rank: 0
Author
Points: 127
Level: System Center Hero
Hi Sameer,



Can u pls post the steps to finding the problem and resolving it?



I am going nuts trying to add new servers on for monitoring and keep getting exactly the symptoms you have described.



Or if u wish to email jDOTbradshawATunswDOTeduDOTau



Thankyou,



John Bradshaw

Reply  
Report Abuse
RE: Agent Installation Issue
Posted: Tue, Sep 29, 2009 9:56 PM :: Rank: 0
Author
Points: 25175
Level: System Center Expert

 John, I just pinged Sameer to see if he has time to share the details on the fix....

Reply  
Report Abuse
RE: Agent Installation Issue
Posted: Wed, Sep 30, 2009 1:58 PM :: Rank: 0
Author
Points: 30
Level: System Center Enthusiast

Can you please post the fix for this. I believe that we are having to same issue.

 

Thanks

 

Reply  
Report Abuse
Re: Agent Installation Issue
Posted: Wed, Sep 30, 2009 3:01 PM :: Rank: 0
Author
Points: 4540
Level: System Center Specialist
Thanks Pete for the email.



JB /gobigblue - I would definitely try the steps which I took in the first post on the top. If no results, then you can try this.

go to the opsmgr database, and open the table MT_UserActionManager . Delete any pending action from that table.

Make sure you take a db backup before performing the above.

Reply  
Report Abuse

Home  » Forums  » Agent Installation Issue
24 Items Page [1]  2      
Tag Cloud
Quick Links
Top Contributors
Pete Zerger
Points: 25175
Level: System Center Expert
Simon Skinner
Points: 20153
Level: System Center Expert
Tommy Gunn
Points: 19054
Level: System Center Expert
MadHatter
Points: 11573
Level: System Center Expert
Stefan Koell
Points: 10134
Level: System Center Expert