www.jayntguru.com » scom

strange .net runtime error and the fix

jayntguru — Wed, 20 Jul 2011 20:02:23 +0000

We had done a POC for Foglight by Quest back in March and ended up not purchasing the product because we already had SCOM in place. After the uninstall and ever since, we have had some issues.

errors logged whenever an app pool starts/restarts containing this description

.NET Runtime version 2.0.50727.3603 – Failed to CoCreate profiler.

.net health monitoring would log an app pool start but not log an app pool stop or shutdown

I had the suspicion that this was related to random app pool restarts during the day on one of our webservices

other debuggers would not attach to an app pool and work (like avicode that comes with SCOM)

I looked for how to fix this since March, of course I was too stubborn to call Quest, so that’s my fault. Much googling didn’t resolve much so when I finally did figure this out, I wanted to post what I found. It turns out that debugging is set in an environment variable. Seems like it’s a session variable because it’s set in the parameters of a service, so that way it runs with the service in the user context of the service. This is something I had never run across before so it seemed kinda odd.

In the end we had to remove the reg key “Environment” (and the contents of the key) from the two locations. This key is what sets the debugger to enabled and tells it which debugger to use.

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W3SVC]
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\IISADMIN]

Once these have been deleted, all you have to do is IISRESET the server and the error is gone and random app pool restarts have ceased. As of this writing I have not tried to reinstall the SCOM AVIcode to the webservice, but I’m certain that it will work now.

AV exclusions

jayntguru — Wed, 25 May 2011 19:19:29 +0000

This comes up often and I can never find everything in one place. This is a good post with a bunch of info and links.

SQL query to tell if SQL logins are using Kerberos or NTLM

jayntguru — Thu, 17 Mar 2011 20:49:15 +0000

Not explanation needed here, if you know what you are looking for, this is it.

SELECT s.session_id , c.connect_time , s.login_time , s.login_name , c.protocol_type , c.auth_scheme , s.HOST_NAME , s.program_name FROM sys.dm_exec_sessions s JOIN sys.dm_exec_connections c ON s.session_id = c.session_id

MSMQ MP for SCOM–has issues..

jayntguru — Mon, 21 Feb 2011 19:19:06 +0000

This is just an example of the kind of stuff I find in the MSMQ MP for scom. Is this alert about the SIZE of the messages or the NUMBER of messages?

scom missing performance counters

jayntguru — Fri, 07 Jan 2011 20:02:57 +0000

I recently had to make some changes to my scom test environment server. It’s a single virtual machine with scom, sql, ssrs, the works on it. Despite what one would think, it has performed well with about 30 agents. Recently I added about that many more and it pushed up the ram to the 4gb max that I had given it. When I added more I realized I had used 2008 x32 for the OS (oops) and was not able to see the extra ram. After some research I found that I could do an upgrade from 2008 standard to enterprise, which is what I did. Afterwards, scom wouldn’t run because the SDK service would not start. This is the event log message.

Log Name:      Operations Manager
Source:        OpsMgr SDK Service
Date:          1/7/2011 2:25:48 PM
Event ID:      26380
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:

Description:
The System Center Operations Manager SDK Service failed due to an unhandled exception.
The service will attempt to restart.
Exception:

System.TypeInitializationException: The type initializer for ‘Microsoft.EnterpriseManagement.Mom.ServiceDataLayer.SdkDataAccess’ threw an exception. —> System.InvalidOperationException: The requested Performance Counter is not a custom counter, it has to be initialized as ReadOnly.
   at System.Diagnostics.PerformanceCounter.Initialize()
   at System.Diagnostics.PerformanceCounter.set_RawValue(Int64 value)
   at Microsoft.EnterpriseManagement.Mom.Internal.PerformanceCounterCategoryDescriptor.SingleInstancePerformanceCounterCategoryManager.CreateCounter(String categoryName, String counterName)
   at Microsoft.EnterpriseManagement.Mom.Internal.PerformanceCounterCategoryDescriptor.SingleInstancePerformanceCounterCategoryManager.CreateCounters(PerformanceCounterCategoryDescriptor categoryInfo)
   at Microsoft.EnterpriseManagement.Mom.Internal.PerformanceCounterCategoryDescriptor.SingleInstancePerformanceCounterCategoryManager..ctor(String instanceName, PerformanceCounterCategoryDescriptor categoryInfo)
   at Microsoft.EnterpriseManagement.Mom.Internal.SingleInstancePerformanceCounterCategoryDescriptor.CreateManager()
   at Microsoft.EnterpriseManagement.Mom.ServiceDataLayer.SdkPerfCounterManager..ctor()
   at Microsoft.EnterpriseManagement.Mom.ServiceDataLayer.SdkDataAccess..cctor()
   — End of inner exception stack trace —
   at Microsoft.EnterpriseManagement.Mom.ServiceDataLayer.SdkDataAccess.Initialize()
   at Microsoft.EnterpriseManagement.Mom.Sdk.Service.SdkSubService.AsyncStartCallback(Object notUsed)
   at System.Threading._ThreadPoolWaitCallback.WaitCallback_Context(Object state)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading._ThreadPoolWaitCallback.PerformWaitCallbackInternal(_ThreadPoolWaitCallback tpWaitCallBack)
   at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback(Object state)
Event Xml:
http://schemas.microsoft.com/win/2004/08/events/event">


    26380
    2
    0
    0×80000000000000

    10025424
    Operations Manager
    INFMGT52.accessgeneral.com


    System.TypeInitializationException: The type initializer for ‘Microsoft.EnterpriseManagement.Mom.ServiceDataLayer.SdkDataAccess’ threw an exception. —> System.InvalidOperationException: The requested Performance Counter is not a custom counter, it has to be initialized as ReadOnly.
   at System.Diagnostics.PerformanceCounter.Initialize()
   at System.Diagnostics.PerformanceCounter.set_RawValue(Int64 value)
   at Microsoft.EnterpriseManagement.Mom.Internal.PerformanceCounterCategoryDescriptor.SingleInstancePerformanceCounterCategoryManager.CreateCounter(String categoryName, String counterName)
   at Microsoft.EnterpriseManagement.Mom.Internal.PerformanceCounterCategoryDescriptor.SingleInstancePerformanceCounterCategoryManager.CreateCounters(PerformanceCounterCategoryDescriptor categoryInfo)
   at Microsoft.EnterpriseManagement.Mom.Internal.PerformanceCounterCategoryDescriptor.SingleInstancePerformanceCounterCategoryManager..ctor(String instanceName, PerformanceCounterCategoryDescriptor categoryInfo)
   at Microsoft.EnterpriseManagement.Mom.Internal.SingleInstancePerformanceCounterCategoryDescriptor.CreateManager()
   at Microsoft.EnterpriseManagement.Mom.ServiceDataLayer.SdkPerfCounterManager..ctor()
   at Microsoft.EnterpriseManagement.Mom.ServiceDataLayer.SdkDataAccess..cctor()
   — End of inner exception stack trace —
   at Microsoft.EnterpriseManagement.Mom.ServiceDataLayer.SdkDataAccess.Initialize()
   at Microsoft.EnterpriseManagement.Mom.Sdk.Service.SdkSubService.AsyncStartCallback(Object notUsed)
   at System.Threading._ThreadPoolWaitCallback.WaitCallback_Context(Object state)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading._ThreadPoolWaitCallback.PerformWaitCallbackInternal(_ThreadPoolWaitCallback tpWaitCallBack)
   at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback(Object state)

After some research I found this article from Microsoft that contains instructions on how to fix this problem.

Dashboarding using the SCCM Dashboard solution accelerator example

jayntguru — Wed, 05 Jan 2011 22:07:25 +0000

I will turn this into a more useful blog post when I have a few minutes but for now it is just a collection of links about dashboarding using the System Center Configuration Manager Solution Accelerator. The short story is that this runs on top of Sharepoint 3.0, which is free, meaning you can run this for free. It’s not specific to SCCM so you can dashboard any sql data you want! Cool!

http://technet.microsoft.com/en-us/library/ff369719.aspx

http://systemscentre.blogspot.com/search/label/Dashboard

http://garyhay.blogspot.com/2010/07/sccm-dashboard-queries-1.html

http://blogs.msdn.com/b/shitanshu/archive/2010/04/08/part-1-how-microsoft-it-using-configuration-manager-dashboard.aspx

http://blogs.msdn.com/b/shitanshu/archive/2010/04/11/part-2-using-configuration-manager-dashboard-for-software-update-deployment-readiness-patch-tuesday-checklist.aspx

http://blogs.msdn.com/b/shitanshu/archive/2010/05/02/part-3-sql-queries-used-for-creating-custom-configuration-manager-dashboard-in-microsoft-it.aspx

http://blogs.msdn.com/b/shitanshu/archive/2010/10/09/part-4-sql-queries-for-creating-configuration-manager-client-health-and-problem-management-dashboard.aspx

http://64.4.11.252/en-us/edge/system-center-configuration-manager-2007-dashboard-part-2-configuration.aspx?query=1

SCOM Windows 2008 R2 Printer MP

jayntguru — Fri, 10 Dec 2010 19:13:52 +0000

Ran across this custom MP by Kevin Holman. Nice! Thank you!

cleaning up the default management pack

jayntguru — Fri, 10 Dec 2010 18:56:05 +0000

SCOM has what I feel is a major bug in that it will allow you to save items (monitors, rules, overrides, etc.) in the default MP. Doing this is bad for a lot of reasons, and not only does SCOM allow you to do this, but it is the default option as well. In my case it turned out that an occasional lack of attention allowed me to do this and then removing MP’s later becomes a huge pain in the rear. Anyway I found this good article on how to clean up the mess.

SCOM: powershell run space failed to start

jayntguru — Tue, 07 Dec 2010 19:07:35 +0000

I have been getting these messages since day 1 and tried various things that didn’t work to resolve.

Below I am pasting an example rule with full text so that if someone is searching they will find it. This is one specific alert, I was having an issue with all non microsoft powershell scripted discoveries. For me this was 99% from the XSNMP SCOM Management Pack. To be clear the MP was not the cause of the problem, only the one that tried to run PS (and not work) the most.

PM

Log Name:
Operations Manager

Source:
Health Service Modules

Event Number:
22400

Level:
1

Logging Computer:

User:
N/A

Description:

Failed to run the PowerShell script due to exception below, this workflow will be unloaded. System.NullReferenceException: Object reference not set to an instance of an object. at System.Environment.GetEnvironmentVariable(String variable, EnvironmentVariableTarget target) at System.Management.Automation.ModuleIntrinsics.GetExpandedEnvironmentVariable(String name, EnvironmentVariableTarget target) at System.Management.Automation.ModuleIntrinsics.SetModulePath() at System.Management.Automation.ExecutionContext.InitializeCommon(AutomationEngine engine, PSHost hostInterface) at System.Management.Automation.AutomationEngine..ctor(PSHost hostInterface, RunspaceConfiguration runspaceConfiguration, InitialSessionState iss) at System.Management.Automation.Runspaces.LocalRunspace.DoOpenHelper() at System.Management.Automation.Runspaces.RunspaceBase.CoreOpen(Boolean syncCall) at Microsoft.EnterpriseManagement.Modules.PowerShell.RunspaceController.RunScript(String scriptName, String scriptBody, Dictionary`2 parameters, PowerShellOutputType outputType, Int32 serializationDepth, IModuleDebug iModuleDebug) at Microsoft.EnterpriseManagement.Modules.PowerShell.PowerShellProbeActionModule.RunScript(RunspaceController runspaceController) Script Name: MemoryPctUtil.ps1 One or more workflows were affected by this. Workflow name: xSNMP.Cisco.Rule.CollectMemoryPoolUtil Instance name: I/O Instance ID: {X} Management group: X

Event Data:

< DataItem type =" System.XmlData " time =" 2010-12-03T19:15:30.1742570-05:00 " sourceHealthServiceId =" X" >

< EventData >

< Data > X

< Data > xSNMP.Cisco.Rule.CollectMemoryPoolUtil

< Data > I/O

< Data > {X}

< Data > MemoryPctUtil.ps1

< Data > 300

< Data > System.NullReferenceException: Object reference not set to an instance of an object. at System.Environment.GetEnvironmentVariable(String variable, EnvironmentVariableTarget target) at System.Management.Automation.ModuleIntrinsics.GetExpandedEnvironmentVariable(String name, EnvironmentVariableTarget target) at System.Management.Automation.ModuleIntrinsics.SetModulePath() at System.Management.Automation.ExecutionContext.InitializeCommon(AutomationEngine engine, PSHost hostInterface) at System.Management.Automation.AutomationEngine..ctor(PSHost hostInterface, RunspaceConfiguration runspaceConfiguration, InitialSessionState iss) at System.Management.Automation.Runspaces.LocalRunspace.DoOpenHelper() at System.Management.Automation.Runspaces.RunspaceBase.CoreOpen(Boolean syncCall) at Microsoft.EnterpriseManagement.Modules.PowerShell.RunspaceController.RunScript(String scriptName, String scriptBody, Dictionary`2 parameters, PowerShellOutputType outputType, Int32 serializationDepth, IModuleDebug iModuleDebug) at Microsoft.EnterpriseManagement.Modules.PowerShell.PowerShellProbeActionModule.RunScript(RunspaceController runspaceController)

< Data />

Eventually the scripts would time out like this:

Time window start
12/7/2010 11:57:38 AM

Time window end
12/7/2010 12:02:37 PM

Time First
12/7/2010 11:57:38 AM

Time Last
12/7/2010 11:57:41 AM

Count
44

Context

Date and Time:
12/7/2010 11:57:41 AM

Log Name:
Operations Manager

Source:
Health Service Modules

Event Number:
22411

Level:
1

Logging Computer:
X

User:
N/A

Description:

The PowerShell script will be dropped because the it has been waiting in the queue for more than 10 minutes. Script Name: DiscoverInterfaceName.ps1 One or more workflows were affected by this. Workflow name: xSNMP.Discovery.InterfaceName Instance name: GigabitEthernet2/21 Instance ID: {X} Management group: X

Event Data:

< DataItem type =" System.XmlData " time =" 2010-12-07T11:57:41.4054873-05:00 " sourceHealthServiceId =" X" >

< EventData >

< Data > X

< Data > xSNMP.Discovery.InterfaceName

< Data > GigabitEthernet2/21

< Data > {X}

< Data > DiscoverInterfaceName.ps1

< Data > 300

< Data > 10

< Data />

The issue turned out to be a permissions issue with the health service since it was trying to run these powershell scripts as local system. This article is the one that finally jogged my memory appropriately. Thank you to the author!

exchange 2010 MP for scom = room for improvement

jayntguru — Tue, 30 Nov 2010 16:09:35 +0000

At first I really liked this MP, it knows a LOT about exchange and there was some serious effort put into making sure it grabs everything. After a while though, there are some things you need to be able to change, but can’t.

Take this alert for disk space, we want to change the % that it alerts on, well guess what you can change with the override?

That’s right, the only thing you can do is enable or disable the rule.. that’s it. And while we’re on the subject of disabling a rule, that’s not working for at least this one:

Here’s a few of the instances of this alert..

And if you look at the overrides, this rule is clearly disabled… but still alerting.

I’m still trying to figure both of these out.