I am currently working on WSUS (windows software update services) here at work, for the most part I’m following this excellent article at Ars. After screwing around with this for much longer than I should have, I was having issues with about half the servers not showing up in the WSUS console. Many things could have been the culprit and I checked them all.. group policy, dns, firewall rules, etc. What was the most frustrating was that I could see the clients touch the WSUS server by looking in the IIS logs, and there were no errors whatsoever, but half the servers wouldn’t show up.
Eventually I realized that it was exactly half of the servers and a light bulb went off. In our environment we have a bunch of web and app servers that are all virtual, and when we build them, we get the first node working right, and then clone and rename the vm to be the redundant node in the farm.This lead me to do some searching and I found this link. Admittedly, this is an old problem, but the first time I have run across it.
The following is a repost of the pertinent bits that have caused my trouble and are the resolution for it.
5. Imaged clients with a duplicate client ID will only appear once in the WSUS Admin Console. Each AU client must have a unique id which is created for each individual install. When imaging systems it is recommended always to use SysPrep. The WSUS admin console will only display one client for each unique ID. If you have multiple clients created from one image which are sharing the same ID, only one will appear in the WSUS admin console. All clients will check in and download updates, but only one will appear and display status in the WSUS admin console. In cases where clients are not checking in, and they were created from images without running SysPrep, the following steps will reset the existing duplicative client IDs.
a. Run regedit and go to HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\WindowsUpdate
b. Delete the PingID, SUSClientID and the AccountDomainSID values
c. Stop and start the Wuauserv Service
d. From the command prompt run: wuauclt /resetauthorization /detectnow
or-
From the command line, once you are sure the AU client is properly configured and not disabled, you could run a batch file (which might look something like this sample) and get the same results:
rem Fixes problem with client machines not showing up on the server due to imaging method
reg delete HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\WindowsUpdate /v AccountDomainSid /f
reg delete HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\WindowsUpdate /v PingID /f
reg delete HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\WindowsUpdate /v SusClientId /f
cls
@echo Triggering detection after resetting WSUS client identity
net stop wuauserv
net start wuauserv
wuauclt /resetauthorization /detectnow
I have finally given in and put my stuff on Twitter. The stuff there will mostly be more tech in nature.
With IIS creating w3wp.exe’s for all of your apps (for which you have created a non default app pool) it is nice, but it’s impossible to tell exactly which is which. Microsoft includes a handy tool to do this with it’s in: %systemroot%\system32\iisapp.vbs (You may need to run it with cscript.exe if that’s not your default for .vbs).
Sample output:
C:\WINDOWS\system32>iisapp
W3WP.exe PID: 2028 AppPoolId: DefaultAppPool
W3WP.exe PID: 20224 AppPoolId: Autotransactiondocs
W3WP.exe PID: 17480 AppPoolId: IRAppPool
C:\WINDOWS\system32>
Oh, this is for Windows Server 2003 (and r2) only.. does not work with 2008 variants.
This seems like a simple thing, really, but every single place I have worked that developed .net code has had this problem, and for some reason, devs don’t seem to get it. (It has been an uphill battle everywhere I have found it with resistance from devs. Why??)
Usually the symptoms start out with scaling issues in production, you’ll see things like threads running out, connections not being closed, and general poor application performance. You can track these down and eventually you will find that in the web.config the setting “debug=enabled” is in there. It is this way because it is the default when you create a .net application. YOU DO NOT LEAVE IT LIKE THIS IN PRODUCTION. When you do, bad things happen, such as timeouts all being set to infinite. Here’s an article with a complete list and some more info.
Depending on whether or not your organization pushes the .configs with the applications (this is a religious discussion in itself.. I recommend not pushing the configs with every build for reasons like this), then when you do your next code push, the process repeats because of this value being set improperly again. The way to fix this for good is that in the production environment, you make a change to the machine.config to enable “retail” mode, which disallows debug from being enabled, regardless of what is in the application’s web.config.
<configuration>
<system.web>
<deployment retail=”true”/>
</system.web>
</configuration>
In my opinion, setting this in the machine.config should be part of the default build/installation/configuration of any production webserver.
This one is really easy, but I am surprised at the number of people who are unable to troubleshoot it. If you think you are having sql connections not being closed properly, just open up perfmon, go to “.net data provider for sqlserver”, and add “numberofreclaimedconnections”. Basically what this counter shows is the number of connections that were not closed by the application, but were closed by the .net garbage collection process. Microsoft’s explanation is on this page. The same counter works for the oracle client as well (except it’s called “oracle” and not “sql”).
The symptoms of this would be a whole lot of idle connections to sql (or oracle) from the webserver, or errors about not being able to obtain a connection, running out of connections in the pool, etc. You can also see lots of connections in the time_wait status if you run a netstat –an.
I had to install the SMTP service on a 2008 server today for the first time, and Microsoft does not make it easy.
- The SMTP install is under features and not under roles.
- When you do install SMTP, where do you manage it, you wonder. Thanks to google and this link, I now know that you have to use the IIS6 manager. What??!!
I haven’t used this one yet but it looks pretty neat. It’s a real release of Microsoft’s IIS web farm deployment tool. Basically it replaces a lot of what I used to use MS application center for and what I do today with batch files and robocopy.
I have spent the better part of the day trying to figure out why a client cert that’s send to our servers just plain out doesn’t work. We get a 403.17 when we shouldn’t. It’s maddening because all we can get out of IIS is the 403.17, and since we do not own the client cert, there’s no easy way for us to test. So we have to trust that our b2b partner has things set up right (we don’t think they do) and then wait 30-35 minutes in between requests for them to re-test. It’s maddening.
This post contains some good troubleshooting info and steps.
Here is a link to some info and a utility for mapping client certs, something that Microsoft inexplicably left out of IIS7. (Why?!)
Another tool I found during this whole process was the IIS7 ssl checker utility. It didn’t (hasn’t) helped our specific situation, but it is something that looks useful that I will save for future problems that will definitely come up.
