Windows EventLog Management–Part 1

Using Event Triggers and Event Forwarding to get what you want from the Event Subsystem

Event logs are horrible, and depending on which log you’re looking at they could be even more horrible!

EventLogs

Seriously though, I shouldn’t say they are horrible, there is just so much that sometimes things get lost in the chatter. Prior to Windows 2008 there were only 3 logs that we had to worry about Application, System and the dreaded Security log. With the Release of Windows Server 2008, countless other logs have been added, Event ID’s have been changed, and the underlying services that report events potentially have their very own log to write to, that is of course if it’s been enabled!

What do you want to know?

  • Low disk space
  • Invalid logon attempts
  • Network outage
  • Service failures
  • Time synchronization issues

The answer to this question depends on so many things, there are literally no wrong answers when it comes to event monitoring. The key is to start looking! You’re never going to know what it is you want to focus your attention on, if you never open up the console.

EventViewer

The nice thing about the Event Viewer is that we can apply filtering so instead of seeing a year’s worth of entries, narrow it down to this month, or this week, or just today. While we would all love to see the friendly blue icon that lets us know the server is happy, the fun only begins when we start looking at the Warning and Error entries. You might be surprised but your log could dwindle from 36,981 entries to a paltry 224!

Granted the resultant log looks way more scary because it’s filled with yellow and red icons, but this view is way more interesting in terms of troubleshooting and monitoring.

Be familiar with the log

  • Are Error and Warning entries all I need to worry about?
  • Is it ok that the Operational log I’m looking at is empty?
  • Do I need to be concerned about each Error or Warning entry?

This goes back to what I said earlier about opening up the console. While it’s ok to start out with a filter for Error and Warning entries, not all logs report problems as an Error or Warning. When you drill down into the Applications and Services logs, often times they are filled with Information entries, and that entry may let you know something either did or didn’t happen. So you need to be familiar with your log and know what events are things you want to be aware of.

For example, the DHCP Filtering log reports MAC denies as an Information entry, if you were filtering for Error and Warning entries you would never see that, assuming you care.

As I mentioned earlier, not all of those new logs are enabled by default. If you have a print server and want to know who is printing, when you open up the Print Server Operational log, it’s empty. Does that mean that nobody is printing? Perhaps, but since the log is off by default, you may want to enable it before you make your decision. Once you become familiar with whatever log you’re looking at, you’ll be able to determine if those red error entries are really something to worry about.

For example, once you have enabled that Print Server Operational log, you may see a recurring Error event, Event ID 812. In our environment our user accounts reside in an external domain, and that entry is indicating more or less a false-positive. The error is the spool file was unable to be deleted, access is denied. The reality is that the spool file did in fact get deleted, so this particular error I don’t need to worry about.

But when I first encountered it, I was concerned about it. I searched for that Event ID on the TechNet site, asked questions in the forums, and searched Google. Only when I satisfied myself that there was nothing I could do to keep this error from occurring, and that the error really wasn’t an error did I decide to ignore it.

Now that we’re comfortable with our logs let look at some fun things to do with them.

Part 1

Part 2

Part 3

Defrag is not defragging

I posted a question in the forums the other day after attempting to defrag one of the drives on my file server. The drive in question is where we have all the application installation files for the School stored.
This particular drive was quite low on free space, so I used the following switches in an attempt to force the defrag to happen anyway.

  • W – Performs full defragmentation. Attempts to consolidate all file fragments, regardless of their size.
  • F – Forces defragmentation of the volume when free space is low.
  • V – Specifies verbose mode. The defragmentation and analysis output is more detailed.

C:Usersjeffpatton.admin>defrag s: -w -f -v
Windows Disk Defragmenter
Copyright (c) 2006 Microsoft Corp.

Defragmentation report for volume S: Software Drive
    Volume size                         = 512 GB
    Cluster size                        = 4 KB
    Used space                          = 450 GB
    Free space                          = 62.40 GB
    Percent free space                  = 12 %

File fragmentation
    Percent file fragmentation          = 55 %
    Total movable files                 = 1,716,060
    Average file size                   = 605 KB
    Total fragmented files              = 1,536
    Total excess fragments              = 1,363,219
    Average fragments per file          = 1.85
    Total unmovable files               = 11

Free space fragmentation
    Free space                          = 62.40 GB
    Total free space extent             = 1,013,736
    Average free space per extent       = 65 KB
    Largest free space extent           = 85 MB

Folder fragmentation
    Total folders                       = 127,411
    Fragmented folders                  = 1
    Excess folder fragments             = 2,819

Master File Table (MFT) fragmentation
    Total MFT size                      = 1.70 GB
    MFT record count                    = 1,777,742
    Percent MFT in use                  = 99
    Total MFT fragments                 = 3

    Note: On NTFS volumes, file fragments larger than 64MB are not included in the fragmentation statistics
Defragmentation report for volume S: Software Drive
    Volume size                         = 512 GB
    Cluster size                        = 4 KB
    Used space                          = 450 GB
    Free space                          = 62.40 GB
    Percent free space                  = 12 %

File fragmentation
    Percent file fragmentation          = 70 %
    Total movable files                 = 1,716,060
    Average file size                   = 605 KB
    Total fragmented files              = 1,533
    Total excess fragments              = 1,363,206
    Average fragments per file          = 1.85
    Total unmovable files               = 11

Free space fragmentation
    Free space                          = 62.40 GB
    Total free space extent             = 1,012,697
    Average free space per extent       = 65 KB
    Largest free space extent           = 85 MB

Folder fragmentation
    Total folders                       = 127,411
    Fragmented folders                  = 1
    Excess folder fragments             = 2,819

Master File Table (MFT) fragmentation
    Total MFT size                      = 1.70 GB
    MFT record count                    = 1,777,742
    Percent MFT in use                  = 99
    Total MFT fragments                 = 3

    Note: On NTFS volumes, file fragments larger than 64MB are not included in the fragmentation statistics
Well, here it is the next day and defrag finished sometime during the night. You will notice now that file fragmentation is where it should be. My assumption from this is that while you can choose the –F option, there is most likely a threshold at which even that won’t work.
C:Usersjeffpatton.admin>defrag s: -w -f -v
Windows Disk Defragmenter
Copyright (c) 2006 Microsoft Corp.

Defragmentation report for volume S: Software Drive
    Volume size                         = 640 GB
    Cluster size                        = 4 KB
    Used space                          = 445 GB
    Free space                          = 195 GB
    Percent free space                  = 30 %

File fragmentation
    Percent file fragmentation          = 45 %
    Total movable files                 = 1,717,746
    Average file size                   = 607 KB
    Total fragmented files              = 446
    Total excess fragments              = 1,159,235
    Average fragments per file          = 1.72
    Total unmovable files               = 11

Free space fragmentation
    Free space                          = 195 GB
    Total free space extent             = 995,580
    Average free space per extent       = 206 KB
    Largest free space extent           = 84.77 GB

Folder fragmentation
    Total folders                       = 127,414
    Fragmented folders                  = 1
    Excess folder fragments             = 0

Master File Table (MFT) fragmentation
    Total MFT size                      = 1.70 GB
    MFT record count                    = 1,778,403
    Percent MFT in use                  = 99
    Total MFT fragments                 = 3

    Note: On NTFS volumes, file fragments larger than 64MB are not included in the fragmentation statistics
Defragmentation report for volume S: Software Drive
    Volume size                         = 640 GB
    Cluster size                        = 4 KB
    Used space                          = 445 GB
    Free space                          = 195 GB
    Percent free space                  = 30 %

File fragmentation
    Percent file fragmentation          = 0 %
    Total movable files                 = 1,717,746
    Average file size                   = 607 KB
    Total fragmented files              = 7
    Total excess fragments              = 305,897
    Average fragments per file          = 1.19
    Total unmovable files               = 11

Free space fragmentation
    Free space                          = 195 GB
    Total free space extent             = 659,294
    Average free space per extent       = 311 KB
    Largest free space extent           = 2.67 GB

Folder fragmentation
    Total folders                       = 127,414
    Fragmented folders                  = 1
    Excess folder fragments             = 0

Master File Table (MFT) fragmentation
    Total MFT size                      = 1.70 GB
    MFT record count                    = 1,778,403
    Percent MFT in use                  = 99
    Total MFT fragments                 = 3

    Note: On NTFS volumes, file fragments larger than 64MB are not included in the fragmentation statistics

SCCM + Dell – ServiceTag = HEADACHE

Well, it’s been a long time since I’ve posted anything and this will most likely be the last post before I hop platforms, but since the resolution took nearly my whole morning I felt it was worthy a posting. As you know we’ve been rolling with System Center Configuration Manager for nearly two years now and while it doesn’t necessarily follow best practices, it’s been as stable as that product appears to be.

So today I was given a machine to image, and I promptly checked our servers to see if it was still lurking out there somewhere.

  • In AD? Nope.
  • In SCCM? Nope.
  • In DHCP? Yes, but that’s ok.

So I fired up the SCCM console and expanded leaf objects until I got down to Computer Association. Right click and choose Import Computer Information and follow through the dialogs. What we’ve always done is add the computer by Name and MAC address, mainly because the GUID is entirely too long for any normal person to remember or have the time to write down. After adding the computer to the proper collection and finishing out of the wizard, I fired up the computer and hit F12, and let the PXE do it’s magic.

Sadly, there was no magic, after about a minute I heard two beeps which means the computer is unable to boot off the network. No worries, I know my view on time is different from that of SCCM so I waited about 5 minutes and tried again, still two beeps. I decided that perhaps I had typoed the MAC address, so decided I’d try it again; two beeps.

That’s when I began my trouble-shooting, the first thing I did was double-check the MAC, as well as make sure there were no duplicates in DHCP, which based on how we roll DHCP is impossible, but still doesn’t hurt to check. There is a report in SCCM that let’s you know if there are duplicate MAC’s inside it’s database, “MAC – Computers for a specific MAC address.” This will let you know if the MAC you enter is associated with more than one computer, it wasn’t.

I fired up the log and saw the following message, I decided to leave the typo intact, since that’s the way Microsoft left it!

The SMS PXE Service Point intructed device to boot normally since it has no PXE advertisement assigned.

Device MAC Address:00:1A:A0:B9:EF:A8 SMBIOS GUID:4C4C4544-0000-2010-8020-80C04F202020.

That seemed odd to me, since I knew that everything was set properly, I decided to restart the Windows Deployment Services (PXE) service. Often times that will fix small issues with PXE booting workstations for imaging, two beeps. That wasn’t it, so then I had to go to the bad place, SMSPXE.LOG. I’m not sure why, but apparently the SMS dev’s decided to punish admins and write absolutely horrible log entries that look like XML and reference line numbers in the source code. Sadly, there wasn’t much of anything different in here either except this:

Device found in the database. MacCount=1 GuidCount=4]LOG]!>

See what I mean by horrible? Anyway, the interesting tidbit is GuidCount=4, wtf? So a while back Carson wrote a report in SCCM that would show GUIDs, I suppose I should post that at some point because it is SUPER handy! But sure enough there were 4 computer’s with the exact same GUID. All of them but one were current, so I decided to nuke them, restart PXE and attempt my boot again, two beeps. I was not a happy camper.

So it was off to Google, since I wasn’t having any luck with the logs. As you can imagine there were lots and lots and LOTS of threads, postings and technical documents.  Most of what I read was from the Microsoft Technet social site, but as I began reading I began noticing that several of these were referencing Dell Optiplex computers. While not the same vintage Optiplex as what I was reading about I was working with Dell hardware nonetheless.

I finally fell on the answer in a two year old thread on the Dell support site. Turns out that service tag is more than just helpful on their website! The GUID for the computer is based on that service tag, and if the motherboard gets replaced and the tech doesn’t add it back into the BIOS, the computer will create a generic one. That’s the GUID or SMSBIOS GUID listed above, and it was a painfully easy fix! You will need to download the ASSET.COM utility from Dell’s Utility FTP site.

Once downloaded run that command with the /S switch and the service tag for your computer. Please be aware that if you muck up that entry, there is no way to remove it! So double, triple, quadruple check before you confirm that it’s ok to update that information.

The only other complicated part for me was finding a floppy drive, and more importantly a floppy disk to make bootable so I could run this and a BIOS update on the intended computer!

HOWTO: Setup IIS 7.5 to use IPv6

I took the SANS 546 class today, and it got me thinking about setting up my server to respond to IPv6 hosts. Steps thus far are pretty straightforward:

  1. Get an account with a tunnel broker
  2. Configure your host
  3. Test connectivity
  4. Configure IIS
  5. Create AAAA record on your DNS provider
  6. Troubleshooting

Tunnel Broker Account

This is very easy and painless! There are several to choose from, but one that was mentioned by the lecturer was Hurricane Electric. Fill out the form and check your email for your password, the whole process takes about 1 minute. Once you login you will need to create your first tunnel:

  1. Login to http://www.tunnelbroker.net/ with your username and password
  2. Click “Create Regular Tunnel”
  3. IPv4 endpoint is your webserver
  4. They will find a tunnel closest to your IP
  5. Click “Create Tunnel”

Configure your host

See? Pretty painless, now that you have your tunnel up you will need to configure your host, since I’m working with Windows 2008 R2, there is actually a set of netsh commands you run. They are specific to your configuration and you can access them by clicking, “Example Configurations” tab on the Tunnel Details page. From the dropdown select your Operating System, and it will give you the commands you need to set it up.

Test connectivity

Once everything is configured the easiest way is attempt to access an IPv6 host:

Configure IIS

Configuring IIS is pretty simple as well, I found that I had some extra stuff that I didn’t think I should need to do though.

  • Open IIS Manager
  • Select the site you wish to enable IPv6 on
  • From the Action pane choose “Bindings…”
  • For basic web server
    • type = http
    • Ip address = ipv6 address
    • port = 80
    • host name = the name you want the server to respond to
  • Click Ok

Create AAAA record on your DNS provider

I use GoDaddy.com for my dns, so you will just need to go into total dns manager and add the AAAA entry. This entry will need to be the same as the host name you specified in your IIS bindings

Troubleshooting

  • The first thing you will want to do is make sure that you are able to ping your own ipv6 address
  • Then try pinging your ipv6 address remotely
  • Repeat these steps with the ipv6 hostname that you set in DNS.
  • It may also be a good idea to visit test-ipv6.com
    • If all those tests fail you may have other issues, that I can’t really help you with

What I found is I was able to ping my ipv6 address locally and remotely, my name ipv6.patton-tech.com resolved locally and remotely, but when pointing a browser at that URL nothing showed up. It was the end of the day when I got this setup, and I had done some of the above basic troubleshooting that all returned successful. This morning I began again I ran the following command:

netstat –s

The output from this command showed several failed attempts over IPv6, this seemed to increase each time I attempted to open the website (could have been coincidence). Since I saw there were failures on IPv6, the next thing I did was run this command:

netstat –an

This should show what addresses are listening on what ports, I saw my IPv4 addresses, but no IPv6. That’s when I started browsing the forums looking for something useful, and I didn’t find much. Most of what I found talked about making sure you didn’t choose the temporary IPv6 address, but since ours is assigned statically via netsh I don’t think that was the problem. Running the command:

netsh interface ipv6 show address

Shows that my interface was a tunnel interface, which makes sense, but that got me spun off into checking the firewall, which wasn’t the problem at all. Finally I found a forum post on iis.net that was close enough to my issue that I was able to resolve it. One of the posters suggested running this command:

netsh http show iplisten

Unlike the original poster I was able to see my IPv4 public address but not my IPv6 address. The suggestion was to remove all iplisten entries which would force iis to listen on all ip addresses. Since I have several services running and listening on port 80, I couldn’t do that. But the syntax of that command led me to TechNet for the proper syntax to add a listener:

netsh http add iplisten ipaddress=ipv6addy

I posted a question in the forums to ask if there is something I have done wrong, or if perhaps the default is to not add the listener but no answer yet. It seems to me that when I add a binding to iis, it should also allow the web server to listen on that address. I know there was nothing I had to do for IPv4, so it’s either a default (not likely) or the fact that this address is set statically (more likely).