Logical Disk Monitor trigger to Hyper-V crash
Home  » Hyper-V  » Logical Disk Monitor trigger to Hyper-V crash

Logical Disk Monitor trigger to Hyper-V crash
Posted: Mon, Dec 14, 2009 2:41 PM :: Rank: 371
Author
Points: 180
Level: System Center Hero

 

We have discovered the root trigger of an issue that others may want to know about. The default settings of the 6.0.667.0, Windows Server 2008 Operating System (Monitoring), Performance, Windows Server 2008 Logical Disk, checks the fragmentation levels of all logical disks on a periodic basis (Every Saturday at 3 a.m. by default). This behavior has caused BSOD, mini dump BUGCHECK_STR: 0x9E on Windows 2008 R2 Data Center servers, Hyper-V Hosts with ISCSI Cluster Shared Volume (CSV). The sequence appears to be that the monitoring of the management pack triggers Event ID: 7036, the Disk Defragmenter service entered the running state, on both Hyper-V Hosts with the Cluster Shared Volume (CSV) Saturday morning at nearly the same time. One of the Hyper-V Host nodes apparently is in control of the CSV and the other node reports Event ID: 5120 (see below). Sometime later the node that has control of the CSV reports Event ID: 1230 and after about 20-30 minutes crashes and reboots.
 
Log Name:      System
Source:        Microsoft-Windows-FailoverClustering
Event ID:      5120
Task Category: Cluster Shared Volume
Level:         Error
Keywords:     
User:          SYSTEM
Computer:      Hyper-VHost01
Description:
Cluster Shared Volume 'Volume1' ('Cluster Disk 1') is no longer available on this node because of 'STATUS_CONNECTION_DISCONNECTED(c000020c)'. All I/O will temporarily be queued until a path to the volume is reestablished.
 
Log Name:      System
Source:        Microsoft-Windows-FailoverClustering
Event ID:      1230
Task Category: Resource Control Manager
Level:         Error
Keywords:     
User:          SYSTEM
Computer:      Hyper-VHost02
Description:
Cluster resource 'Cluster Disk 1' (resource type '', DLL 'clusres.dll') either crashed or deadlocked. The Resource Hosting Subsystem (RHS) process will now attempt to terminate, and the resource will be marked to run in a separate monitor.
 
At this point we have created an override with a group containing the Hyper-V Host and set it to disable the monitor Logical Disk Fragmentation Level for the group.
 
Reference: Why is my 2008 Failover Clustering node blue screening with a Stop 0x0000009E?
   Report Abuse
Re: Logical Disk Monitor trigger to Hyper-V crash
Posted: Tue, Dec 22, 2009 7:18 PM :: Rank: 137
Author
Points: 7471
Level: System Center Specialist
You are suggesting this occurs predictably when the disk defragmentation recovery task it executed? Have you attempted to change the HangRecoveryAction to a value that will not perform the memory dump (such as 1 or 2)?
   Report Abuse
RE: Logical Disk Monitor trigger to Hyper-V crash
Posted: Wed, Dec 23, 2009 4:55 PM :: Rank: 126
Author
Points: 180
Level: System Center Hero

We were experiencing a Host crash every Saturday at 3 am for a couple of weeks and tracked it down to the Windows Server 2008 Logical Disk Monitor triggering a check of the logical disks fragmentation level at this default day time.  When an override was created to move the fragmentation check task running time the crash moved.  In order to keep our production hosts running we have disabled the Logical Disk Fragmentation Level Monitor from running on the Hyper-V R2 Hosts.  This was not running a recovery of the fragmentation but only the check on a 1.4TB ISCSI Cluster Shared Volume (CSV). On another Hyper-V R2 CSV with a much smaller volume the check would complete within minutes and not trigger the crash.  Wanted anyone else out there to be able to discover our findings if searching by the events that were being logged.

   Report Abuse

Home  » Hyper-V  » Logical Disk Monitor trigger to Hyper-V crash
Top Contributors
Featured Members
Pete Zerger
Points: 65502
Level: System Center Expert
Tommy Gunn
Points: 42718
Level: System Center Expert
Simon Skinner
Points: 40744
Level: System Center Expert
Stefan Koell
Points: 28999
Level: System Center Expert
Andreas Zuckerhut
Points: 27584
Level: System Center Expert