Having been an avid Nagios/Opsview user for a while I am always keen to see new features that make my life of defining and managing systems easier. I had been meaning to try out the host attributes feature of Opsview for a while to redefine the way I monitor various “generic” features on my infrastructure. Up until now I have had to create an exception for a host that I want to monitor in a slightly different way and remembering what did/didnt have exceptions was never the easiest thing to do.

This has all changed with the Host Attributes feature in Opsview. I can now define a single service check that will take a number of values (currently Opsview 3.7.2 will only let you define one however looking at the SQL database there is capacity for 9 arguments. A forum post from Ton Voon has revealed a patch to the host-attributes tab that allows you to define 4 attributes which should be released in an upcoming release – 3.7.3 maybe). This means that I can define a host attribute (e.g. DISK) and then set in this the partition/disk name and the warning/critical values in different arguments to make sure that I can reduce the number of custom service checks or exceptions that I need to define.

I have managed to abstract my Disk space checks and also some checks for Exchange Information Store sizes across my organisation. I plan to try and further abstract other generalised items of monitoring (e.g. Windows Services, Performance counters etc).

Once I had created these checks I needed to add in a viewport to display the status of my Information Stores. In the past this used to be setup individually on each host and service check manually. In the latest release its possible to create a new keyword and then add in the host/services that you want from the Keywords tab. This has made the process of making new views/displays easier and made the monitoring much simpler.

When I get some time I will put up some pictures to go with this article and expand on my ability to monitor network interfaces with the latest version of Opsview.

abstraction · attributes · monitoring · nagios · opsview · snmp

Just spent a few days getting RANCID setup on one of my monitoring servers to backup my device configs on a daily basis. Whilst setting it up I followed a number of guides to get my config files setup and checked. The one thing I couldnt get to work however was the email when RACID detected a config change on one of the network devices.

Scouring the Internet I couldnt find what I had missed. Postfix was setup correctly and I could use the aliases I setup in /etc/alises if i “telnet localhost 25″ and mail was delivered. In the end looking at the update logs I could see a line saying it couldnt find sendmail.

A quick look at racnid_control and I updated the lines that referenced sendmail to include a full path to /usr/sbin/sendmail and low and behold my inbox was full of config changes this morning.

I’m sure that if I was able to get the money to buy Opsview Enterprise I would make full use of the RANCID module within this but for the moment this works well enough for me.

My next goal is to get SNMP Trap processing setup so that if the appropriate trap is received from a monitored device it will pull the latest config down and we will always have the latest config.

cisco · config · monitoring · rancid · routers · switches · wireless

Very un-work related but this was filmed outside my old office a few years ago and was sent round work earlier today. Well worth a watch and a chuckle!

No tags

I have just come across an interesting issue with my production Opsview server where the web interface was loading successfully on http://<server>:3000 however http://<server> through the apache proxy was not working following an upgrade from Ubuntu 8.04 to 10.04

The upgrade process went smoothly and everything looked OK having reinstalled Opsview (the upgrade process will uninstall the package but the data is kept in the databases) except for the fact that I couldnt expand any of the menus across the top of Opsview and a small box had appeared under the search bar.

A bit more research in my browser showed the following errors:

Following on from this I popped a quick email to the opsview-users distribution group. to find out if they were aware of any issues with the upgrade process. I reviewed the Apache proxy config and replaced this with the stock one from the new Opsview install. This didnt help.

Next I ran through disbaling and re-enabling the three proxy modules and reloading apache a few times and still no joy.

Feedback from the mailing list suggested that the proxy config was correct but to try accessing the javascript file http://<server>/javascript/prototype.js (this returned a 404 error) and to also look at the apache error logs at the same time.

The logs from apache gave me the following:

[Thu May 27 10:19:35 2010] [error] [client 10.9.4.138] File does not exist: /usr/share/javascript/prototype.js
[Thu May 27 10:20:08 2010] [error] [client 10.9.4.138] File does not exist: /usr/share/javascript/prototype.js, referer: http://10.9.0.220/status/hostgroup
[Thu May 27 10:20:08 2010] [error] [client 10.9.4.138] File does not exist: /usr/share/javascript/scriptaculous.js, referer: http://10.9.0.220/status/hostgroup
[Thu May 27 10:20:08 2010] [error] [client 10.9.4.138] File does not exist: /usr/share/javascript/validation.js, referer: http://10.9.0.220/status/hostgroup
[Thu May 27 10:20:08 2010] [error] [client 10.9.4.138] File does not exist: /usr/share/javascript/forms.js, referer: http://10.9.0.220/status/hostgroup
[Thu May 27 10:20:08 2010] [error] [client 10.9.4.138] File does not exist: /usr/share/javascript/opsview_sidenav.js, referer: http://10.9.0.220/status/hostgroup
[Thu May 27 10:20:08 2010] [error] [client 10.9.4.138] File does not exist: /usr/share/javascript/anylinkcssmenu.js, referer: http://10.9.0.220/status/hostgroup
[Thu May 27 10:20:08 2010] [error] [client 10.9.4.138] File does not exist: /usr/share/javascript/overlib_mini.js, referer: http://10.9.0.220/status/hostgroup
[Thu May 27 10:20:08 2010] [error] [client 10.9.4.138] File does not exist: /usr/share/javascript/status.js, referer: http://10.9.0.220/status/hostgroup
[Thu May 27 10:20:08 2010] [error] [client 10.9.4.138] File does not exist: /usr/share/javascript/prototype.js, referer: http://10.9.0.220/status/hostgroup
[Thu May 27 10:20:08 2010] [error] [client 10.9.4.138] File does not exist: /usr/share/javascript/scriptaculous.js, referer: http://10.9.0.220/status/hostgroup
[Thu May 27 10:20:08 2010] [error] [client 10.9.4.138] File does not exist: /usr/share/javascript/validation.js, referer: http://10.9.0.220/status/hostgroup
[Thu May 27 10:20:08 2010] [error] [client 10.9.4.138] File does not exist: /usr/share/javascript/forms.js, referer: http://10.9.0.220/status/hostgroup
[Thu May 27 10:20:08 2010] [error] [client 10.9.4.138] File does not exist: /usr/share/javascript/opsview_sidenav.js, referer: http://10.9.0.220/status/hostgroup
[Thu May 27 10:20:08 2010] [error] [client 10.9.4.138] File does not exist: /usr/share/javascript/anylinkcssmenu.js, referer: http://10.9.0.220/status/hostgroup
[Thu May 27 10:20:08 2010] [error] [client 10.9.4.138] File does not exist: /usr/share/javascript/overlib_mini.js, referer: http://10.9.0.220/status/hostgroup
[Thu May 27 10:20:08 2010] [error] [client 10.9.4.138] File does not exist: /usr/share/javascript/status.js, referer: http://10.9.0.220/status/hostgroup

I would expect the path for the javascript to be /usr/local/nagios/share/javascript/… and not just /usr/share/javascript. I double checked my apache config and ran through all the configuration files that were included. excluded the /etc/apache2/conf.d directory and reloaded Apache. The result… Opsview loaded and displayed correctly:

Going back through the different files in the directory I came across javascript-common.conf which has the following code in it:

Alias /javascript /usr/share/javascript/

<Directory "/usr/share/javascript/">
        Options FollowSymLinks MultiViews
</Directory>

I removed the symlink, re-enabled the conf.d directory in the apache config and all looked good.

Having a quickl look round I couldnt find any reason for the package being installed on my machine so I removed it and restarted apache followed by an apt-get check to see if there were any broken dependencies and there were none.

Upshot of all of this… Unless you want all your Javascript to be in one location dont install the javascript-common package.

10.04 · 8.04 · javascript · javascript-common · opsview · ubuntu

… since I wrote anything here and that is because i have been quite busy over the past few months. I thought I would put a quick post up on what I am doing at the moment.

In a few days I am moving from the leafy town of Leatherhead up to the “big smoke” that is London. I have spent a fair bit of time getting this sorted and looking forward to move. When I have settled in I will need to get my various network aspects updated and my “lab” will become a much more complicated setup. Planning of this is still underway.

I am also starting to look at a number of different aspects of how I can keep my networks monitored and protected as best I can. At the moment I am looking at deploying a RANCID server to maintain config files on various network devices that I am responsible for. The initial setup is not that complicated however I am looking at a way of integrating the scheduled updates with SNMP trap processing so that when a user logs out of a device the trap is sent back to the network and initiates rancid to connect to the device and pull down the current config. I am also looking at how I can log the user that makes the config changes as part of a future project to look at tieing RADIUS/IAS authentication into accessing the network devices.

There is still the ongoing interest in monitoring and I am looking at how I can further manage and maintain my opsview configurations and keep track of my network infrastructure.

Anyway – got to go. Just seen the end of 24 and need to get some food.

No tags

I’ve been having some issues with my clients’ Blackberry handsets this morning and just received the following update from T-Mobile

I am afraid we have currently got an issue with 2g data services which is affecting all BlackBerry. Its currently affecting central and east London.

I dont have any further information at this time but it looks to only be affecting the BB Curves we have and not the Bolds. Looking at a couple of devices they appear to have GPRS back again but I am still waiting on the OK from T-Mobile to confirm everything is working

2G · 3G · BES · BIS · blackberry · Mobile · T-Mobile

Looking out of the window this morning the sky is blue and there isn’t a cloud to be seen in the sky and this has reminded me of Tit Monday….

Ah, Tit Monday. It’s not that far off now, that glorious day when, heading into work on the bus, or walking to the Tube, or sitting on the train, you find yourself suddenly chirpier than you have been in months.

You find yourself smiling at strangers again. There is a mild involuntary tumescence in your trousers that comes and goes throughout the morning with the comforting regularity of a heartbeat.

And then you get a text around lunchtime from a mate which says: “At last, Tit Monday!” And you instantly understand why you are so happy. For Tit Monday is that special day in the year when, for the first time, the temperature rises above that magical point which causes girls getting dressed in the morning to decide to show a bit of skin.

After months of dull colours and chunky knit, the world’s birds suddenly dive into last summer’s wardrobe (they’ve not had chance to buy this season’s stuff) and chuck it on without a thought.

Your urban landscape is suddenly lightened with acres of naked arm and leg and, after many dark months of burrowing, breasts rising to the surface like moles at dusk. Big breasts in white work shirts straining
at the buttons. Small breasts braless in vest tops, the nipples fretted by ribby fabrics. Breasts in summer dresses bouncing in the distance so that they catch your eye before you even notice there is someone wearing
them. Breasts nudging out from the crowd at traffic islands, quivering to cross the road…And you know it is nearly summer.

For previous generations, the arrival of spring was heralded by the sound of the first cuckoo. For us, it is Tit Monday. Not that it always falls on a Monday. Like Easter, Tit Monday is a moveable feast. Last
year it fell on a Friday. Friday 29 April, to be precise, when temperatures maxed out at 22.1C after nothing much above 16C all year. It last fell on a Monday in 2004, when temperatures leapt to 22C on 24
April. And then, of course, there is Tit Monday Night.

You see, in early summer, temperatures drop off very dramatically when night falls (Tit Friday 2005 dropped away to a parky 11.8C). But the dollies are not prepared. Slightly stunned by the morning heat, they
drag out the summer clothes but forget to bring a cardie (a mistake they will not make again until next year), so that when they’re all standing outside All Bar One after work celebrating the arrival of spring, their
barely covered nipples have no protection from the cold.

It’s like a Bring-and-Buy sale where everyone has brought hat pegs. It’s like a prog-rock gig where, instead of lighters, everyone is holding up nipples.

So when will Tit Monday fall this year? Will you be the first to text your mates with the announcement? Do not shoot your bolt too early. There will be false starts. You will smell fresh cut grass and see a couple of early starters and feel compelled to declare Tit Monday. But your more level-headed friends will tell you to hold your horses, keep your powder dry, don’t fire until you see the whites of their bra straps

As the poet said: one bold Northern slapper in a bikini doth not a summer make.

comedy · Fun · summer · titmonday · weather

Today I sat and passed, after a long time of putting it off, my JNCIA (Juniper Networks Certified Internet Associate) Firewall/VPN Exam.

This now means that I have a qualification in the firewall technology that we are using at work. Hopefully I can play with some of the more funky stuff they use and work towards my JNCIS now

exam · juniper · qualification

Following on from my Symantec AV check I have written a first version of a similar check for E-Trust virus definitions. The format and structure to the check is the same as this check but it should return the relevant information for Computer Assoicates E-Trust Antivirus product.

For details on installation and configuration please check out the previous post. For the source code please check out the details below. If you wish to download this from Monitoring Exchange please use this link.

' Script: check_etrust_av.vbs
' Author: Matt White
' Version: 1.0
' Date: 12-03-2010
' Details: Check the current definitions for E-Trust AntiVirus are within acceptable bounds
' Usage: cscript /nologo check_etrust_av.vbs -w: -c:

' Define Constants for the script exiting
Const intOK = 0
Const intWarning = 1
Const intCritical = 2
Const intUnknown = 3

' Parse Arguments to find Warning and Critical Levels
If Wscript.Arguments.Named.Exists("w") Then
  intWarnLevel = Cint(Wscript.Arguments.Named("w"))
Else
  intWarnLevel = 2
End If

If Wscript.Arguments.Named.Exists("c") Then
  intCritLevel = Cint(Wscript.Arguments.Named("c"))
Else
  intCritLevel = 4
End If

' Define Date Regular Expression
Const strDateRegExp = "(0[1-9]|1[012])[- /.](0[1-9]|[12][0-9]|3[01])[- /.](19|20)\d\d"

' Create required objects
Set objShell = CreateObject("Wscript.Shell")
Set ObjProcess = ObjShell.Environment("Process")
Set objRegExp = New RegExp
Set objReg=GetObject("winmgmts:{impersonationLevel=impersonate}!\\.\root\default:StdRegProv")

const HKEY_CURRENT_USER = &H80000001
const HKEY_LOCAL_MACHINE = &H80000002

' read the path of E-Trust Anti-Virus from the registry
strKeyPath = "SOFTWARE\ComputerAssociates\ScanEngine\Path"
objReg.GetStringValue HKEY_LOCAL_MACHINE,strKeyPath,"Engine",strScanEnginePath

If TypeName(StrScanEnginePath) = "Null" Then
  WScript.Echo "UKNOWN: Cannot read registry Info. Is E-Trust installed?"
  Wscript.Quit(intUnknown)
End If

'strScanEnginePath = ObjShell.RegRead("HKLM\SOFTWARE\ComputerAssociates\ScanEngine\Path\Engine")

' Determine CPU architecture for correct executable to run
strCPUArch = objProcess("PROCESSOR_ARCHITECTURE")
If InStr(1, strCPUArch, "x86") > 0 Then
strExecutable = "\inocmd32.exe"
ElseIf InStr(1, strCPUArch, "64") > 0 Then
strExecutable = "\inocmd64.exe"
End If

' If the path doesnt exist Exit with an Unknown status
If Len(StrScanEnginePath) = 0 Then
  Wscript.Echo "UNKNOWN: Unable to read registry path"
  Wscript.Quit(intUnknown)
End If

' Run the command and read the output into a string
Set objExec = objShell.Exec(strScanEnginePath & strExecutable & " /sig")
strVirusDefs = objExec.StdOut.ReadAll()

' Search the Virus definition for the date using Regular Expression
objRegExp.Pattern = strDateRegExp
objRegExp.Global = True
objRegExp.IgnoreCase = True
Set regExpMatch = objRegExp.Execute(strVirusDefs)

' If date not found in the output. Exit with a warning
If regExpMatch.Count = 0 Then
  Wscript.Echo "UNKNOWN: Unable to read date from the output"
  Wscript.Quit(intUnknown)
End If

intDateDifference = DateDiff("d",CDate(regExpMatch(0).Value), Date)

Wscript.Echo strVirusDefs
If intDateDifference > intCritLevel Then
  Wscript.Quit(intCritical)
ElseIf intDateDifference > intWarnLevel Then
  Wscript.Quit(intWarning)
ElseIf intDateDifference <= intWarnLevel Then
  Wscript.Quit(intOK)
End If
Wscript.Quit(intUnknown)

Anti-Virus · CA · nagios · NRPE · opsview

I had a small shock this evening when I noticed that one of the iSCSI mounts to my lab servers was not working as expected. I could see a folder structure but no data in the folder. I have had issues in the past because the iSCSI mount is a dynamic disk (Yes I know now that I should have left it a basic disk but I have not got enough space to move the relevant data off, covert to basic and move it back again) and when I reboot the server that it mounts to I have to reactivate the disk manually and recreate the appropriate shares. This issue was different.

I logged onto the admin interface for my NAS (Thecus 5200BR Pro) and checked the disk status to find the following screen

I hadnt been notified that my Nas was not 100% healthy so this was quite a shock. Clicking further on the Warning I had the following screen confront me

I think this is why I have some errors on my server. I shut down all the VMs and powered the NAS back on to do a file system check which it seemed to pass but it only checks the file system and not the iSCSI mounts that exist.

I rebooted the NAS again into normal operating mode and powered my ESX servers back on, logged back into my server with the iSCSI mount and reactivated the local disk and shared the folders again. Data was there

I dont trust that this wont happen again so I have purchsed a further two 1TB Western Digital hard drives from my preferred supplies (www.overclockers.co.uk) and am having them shipped to work so I can get them installed as soon as possible in my lab. I think I am also going to take this as a chance to move the iSCSI mount away from the existing setup and onto a new iSCSI array that is mounted on a dedicated iSCSI LAN and hopefully improve performance a little bit.

failed disk · iSCSI · NAS · RAID · Thecus · Western Digital

As I start to write/modify more checks and scripts for monitoring applications in Nagios/Opsview I have decided to share these as much as possible with the community so they can enjoy, and if necessary, improve the scripts I have written. I have decided to use the MonitoringExchange.org website to host my scripts (as well as detailing them on this blog) as I have found a number of good scripts here that do what I wanted them to.

All the scripts should appear as projects under my profile (wibble) with a link back to the same script on the blog here.  I will also endeavour to post the link to Monitoring Exchange in the bottom of the blog post.

monitoring · MonitoringExchange · nagios · opsview

This morning whilst deploying a modified version of the Symantec Anti-Virus check from MonitoringExchange.org I noticed that on my 64-bit hosts that the check was not returning the correct data and instead of the expected output I was receiving the following error code:

check_av.vbs(51, 1) Microsoft VBScript runtime error: Type mismatch: 'strValue'

Initially I thought this could be a change due to the new installs being Symantec Endpoint Protection compared to the previous times I had implemented this using Symantec Anti-Virus 10.x but the SEP installs on the 32-bit systems were working fine however the 64-bit versions were not.

A quick look in the registry showed me that the value that is read by the script is not there on the 64-bit version and has been moved to another location (HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Symantec\SharedDefs\DefWatch). I sat down with the script and quickly wrote in some extra code that would allow me to change the search path depending on the Operating System Architecture. I also added in some more error checking so if the key didnt exist then rather than exiting with an OK status it returns an UNKNOWN status and a relevant error message.

As I use NSClient++ to enable me to monitor my Windows servers I simply save the script to the NSClient++\scripts folder and add the following line into my NSCI.ini under [NRPE Handlers]

check_av=cscript.exe //NoLogo scripts\check_av.vbs /W:$ARG1$ /c:$ARG2$

Then from within Nagios or Opsview define the command for check_nrpe

check_nrpe -H $HOSTADDRESS$ -c check_av -a 2 3

The full script is listed below and is also available on Monitoring Exchange (link):

' Script: check_av.vbs
' Author: Matt White
' Version: 1.1
' Date: 01-03-2010
' Details: Check the current definitions for Symantec AntiVirus are within acceptable bounds
' Usage: cscript /nologo check_av.vbs -w:<days> -c:<days>

' Define Constants for the script exiting
Const intOK = 0
Const intWarning = 1
Const intCritical = 2
Const intUnknown = 3

' Create required objects
Set ObjShell = CreateObject("WScript.Shell")
Set ObjProcess = ObjShell.Environment("Process")

const HKEY_CURRENT_USER = &H80000001
const HKEY_LOCAL_MACHINE = &H80000002

Dim strKeyPath, strSymantecVer
Dim intWarnLevel, intCritLevel, intYear, intMonth , intDay, intVer_Major, intDateDifference
Dim year, Month , Day, Ver_Major
Dim arrValue

' Parse Arguments to find Warning and Critical Levels
If Wscript.Arguments.Named.Exists("w") Then
intWarnLevel = Cint(Wscript.Arguments.Named("w"))
Else
intWarnLevel = 2
End If

If Wscript.Arguments.Named.Exists("c") Then
intCritLevel = Cint(Wscript.Arguments.Named("c"))
Else
intCritLevel = 4
End If

' Determine CPU architecture for correct location of the registry key
strCPUArch = objProcess("PROCESSOR_ARCHITECTURE")
If InStr(1, strCPUArch, "x86") > 0 Then
strKeyPath = "SOFTWARE\Symantec\SharedDefs\DefWatch"
ElseIf InStr(1, strCPUArch, "64") > 0 Then
strKeyPath = "SOFTWARE\Wow6432Node\Symantec\SharedDefs\DefWatch"
End If

' Query Registry using WMI to obtain the definition value
Set oReg=GetObject("winmgmts:{impersonationLevel=impersonate}!\\.\root\default:StdRegProv")
oReg.GetBinaryValue HKEY_LOCAL_MACHINE,strKeyPath,"DefVersion",arrValue

' If the query doesnt return an array Quit - Unknown
If isArray(arrValue) = vbFalse Then
Wscript.Echo "UNKNOWN - Unable to read Definitions from the Registry"
Wscript.Quit(intUnknown)
End If

' Generate output from the registry value
intYear = CLng("&H" & hex(arrValue(1)) & hex(arrValue(0)))
intMonth = CLng("&H" & hex(arrValue(3)) & hex(arrValue(2)))
intDay = CLng("&H" & hex(arrValue(7)) & hex(arrValue(6)))
intVer_Major = CLng("&H" & hex(arrValue(17)) & hex(arrValue(16)))
strSymantecVer= intYear & "-" & intMonth & "-" & intDay & " rev. " & intVer_Major
intDateDifference = DateDiff("d", intYear & "/" & intMonth & "/" & intDay, Date)

' Output current version and definition age as Performance data
Wscript.Echo("Current Definitions: " & strSymantecVer & " Which are " & intDateDifference & " days old" & "|age=" & intDateDifference)

If intDateDifference > intCritLevel Then
Wscript.Quit(intCritical)
ElseIf intDateDifference > intWarnLevel Then
Wscript.Quit(intWarning)
ElseIf intDateDifference <= intWarnLevel Then
Wscript.Quit(intOK)
End If
Wscript.Quit(intUnknown)

Anti-Virus · nagios · NRPE · opsview · Symantec

Whilst trying to make the home page and blog page look the same I managed to break the wp-admin section of my blog. Thanks to the guys at Loho.co.uk they have migrated my whole site to a new platform where I can administer the site more efficiently and I have been able to remove the faulty theme. Hopefully I can fix what I broke but for now its going to be the standard WP theme.

Blog · Theme · WordPress

If you try to synchronise a Windows Mobile PDA with Exchange Direct Push using SSL and the certificate is not issued by a Certification Authority (CA) that is in the PDA’s trusted certificate list then the device will not activate. Most commonly I have come across this with SBS servers that use the default self-signed certificate.

The solution should always be to purchase and install a certificate that is issued by a trusted CA to overcome the issue and the PDA will start to work automatically in these cases. If however you don’t want to purchase the certificate then you can bypass the security checks that Windows Mobile imposes on Active Sync. To do this requires you to install the certificate on the PDA and modify the registry to accept the installed certificate as a trusted one.

As each time I have done this I haven’t had the relevant PDA in front of me I have found a useful tool, that saves you trying to talk the end user through making the changes themselves, called My Mobiler (http://mymobiler.com/) which lets you interact with the PDA from your desktop.

1. Install the certificate on the PDA
   1. Browse to your Outlook Web Access URL in Internet Explorer and save the certificate locally to your desktop by clicking on the padlock icon
   2. Connect the PDA via USB to the PC and allow Active Sync to connect.
   3. Click Explore Device in Active Sync and copy the certificate to the folder that is open
   4. Open File Explorer on the PDA and click on the certificate (it should be in My Documents)
   5. You will likely receive errors that the certificate is not trusted. Click More and then Install
   6. You should receive confirmation the certificate has been installed successfully.
2. Install PHM RegEdit on your PDA
   1. There are a number of places to download the .cab file on the Internet (link) save this to your desktop
   2. With the PDA connected Explore the device again and copy the .cab file to the device
   3. Open File Explorer and click on the .cab to install it (again it should be in My Documents)
   4. When prompted that the installer cannot be verified click Install
3. Apply the registry fix
   1. Click Start and select Programs. Scroll down and click on PHM Registry Editor
   2. Expand the following path: HKEY_CURRENT_USER\SOFTWARE\Microsoft\ActiveSync\Partners
   3. You will see a list of GUID keys. Search through these for the one that contains the name “Microsoft Exchange” this is the key you need to modify
   4. Click Edit and select new DWORD
   5. Name the DWORD “Secure” and leave the value as 0
   6. Exit the Registry Editor

If everything has worked correctly your PDA should now synchronise with Exchange

active sync · certificate · exchange · pda · windows mobile · winnmo

Looked at an issue a colleague had today where the SendAs permissions for a user were being removed automatically from a their account causing issues with their PA not being able to send email as they had configured it. The problem here was that the user in question was in one of the protected AdminSDHolder groups and Active Directory will reset the Send As permissions for members of these groups on an hourly basis.

As well as the ability for another user to Send-As the user in question this can also have implications if you run Blackberry Enterprise Server as the BES Service Account needs Send-As permissions to forward email from a handset to another recipient.

Microsoft have released a KB article on this (907434) which details the situation further but basically the solution should be to remove the user from the groups and if they need to perform the actions granted by the AdminSDHolder groups then they should be given a second “admin” account to perform these tasks.

The list of groups that are affected by the AdminSDHolder changes are:

• Administrators
• Account Operators
• Server Operators
• Print Operators
• Backup Operators
• Domain Admins
• Schema Admins
• Enterprise Admins
• Cert Publishers

AdminSDHolder · blackberry · permissions · send As

Last night I wrote an article about how to monitor the health of an ESXi server (link here) and I wanted to explain a bit more about my findings with SNMP on an ESXi host.

My goal with the monitoring was to use the check_dell and check_hp commands I have found for Nagios/Opsview to monitor the hardware that ESX is running on. The ESXi installs I am working with are using the Dell and HP management agents installed so I thought that everything would work out of the box and enabling SNMP would let me query the different aspects of the hardware.

The official line from VMWare was that SNMP is not enabled on ESXi and with no console cant be enabled. I knew however, having read a recent post on the TechHead blog (link here) that you could see the snmp.xml file and this shows that it is not enabled which made me think it must be possible to enable it. I was right.

A quick google came up with this article and I had a look and this was a fairly simple process to run:

First you need to enter the “unsupported” console on your ESXi server. To do this press Ctrl+Alt+F1 at your ESX console, now type the word unsupported (N.B. you will not see the text on your screen) and press Enter. If all goes well you should see a password prompt, enter your root password here and you should get a warning you are entering a mode that should only be enabled with VMWare support and be presented with a console.

type the following command to enter the VI text editor and start to modify the snmp.xml file:

vi /etc/vmware/snmp.xml

You should see a single line of text at the top of the screen which is the contents of the xml file. Press i to enter Insert mode and change

<enabled>false</enabled>

to

<enabled>true</enabled>

Then scroll across and add the community name you want the SNMP agent to respond on and place this between the following tags

<communities></communities>

so it should look like

<communities>public</communities>

I wasnt interested in setting up SNMP traps so left this blank and quit the VI editor by press Esc to exit insert mode and then :wq to write the file and quit the editor.

Finally we need to restart the services on the esx host which can be done with the following command

/sbin/services.sh restart

Great, SNMP is now enabled so I should be able to get the information from the HP/Dell management agents that I want. Wrong. My snmpwalk of the host provided little to no useful information about what I was trying to unlock.

opsview@LON-SVR-MON1:~$ snmpwalk -v 2c -c public 10.9.0.65
SNMPv2-MIB::sysDescr.0 = STRING: VMware ESX 4.0.0 build-219382 VMware, Inc. x86_64
SNMPv2-MIB::sysObjectID.0 = OID: SNMPv2-SMI::enterprises.6876.4.1
DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (6061646) 16:50:16.46
SNMPv2-MIB::sysContact.0 = STRING: not set
SNMPv2-MIB::sysName.0 = STRING: lon-svr-esx2.domain.local
SNMPv2-MIB::sysLocation.0 = STRING: not set
SNMPv2-MIB::sysServices.0 = INTEGER: 72
SNMPv2-MIB::sysORLastChange.0 = Timeticks: (0) 0:00:00.00
SNMPv2-MIB::sysORID.1 = OID: SNMPv2-MIB::snmpMIB
SNMPv2-MIB::sysORID.2 = OID: IF-MIB::ifMIB
SNMPv2-MIB::sysORID.3 = OID: SNMPv2-SMI::enterprises.6876.1.10
SNMPv2-MIB::sysORID.4 = OID: SNMPv2-SMI::enterprises.6876.2.10
SNMPv2-MIB::sysORID.5 = OID: SNMPv2-SMI::enterprises.6876.3.10
SNMPv2-MIB::sysORDescr.1 = STRING: SNMPv2-MIB, RFC 3418
SNMPv2-MIB::sysORDescr.2 = STRING: IF-MIB, RFC 2863
SNMPv2-MIB::sysORDescr.3 = STRING: VMWARE-SYSTEM-MIB, REVISION 200801120000Z
SNMPv2-MIB::sysORDescr.4 = STRING: VMWARE-VMINFO-MIB, REVISION 200810230000Z
SNMPv2-MIB::sysORDescr.5 = STRING: VMWARE-RESOURCES-MIB, REVISION 200810150000Z
SNMPv2-MIB::sysORUpTime.1 = Timeticks: (0) 0:00:00.00
SNMPv2-MIB::sysORUpTime.2 = Timeticks: (0) 0:00:00.00
SNMPv2-MIB::sysORUpTime.3 = Timeticks: (0) 0:00:00.00
SNMPv2-MIB::sysORUpTime.4 = Timeticks: (0) 0:00:00.00
SNMPv2-MIB::sysORUpTime.5 = Timeticks: (0) 0:00:00.00
IF-MIB::ifNumber.0 = INTEGER: 4
IF-MIB::ifDescr.1 = STRING: Device vmnic0 at 02:00.0 bnx2
IF-MIB::ifDescr.2 = STRING: Device vmnic1 at 02:00.1 bnx2
IF-MIB::ifDescr.3 = STRING: Device vmnic2 at 03:00.0 bnx2
IF-MIB::ifDescr.4 = STRING: Device vmnic3 at 03:00.1 bnx2
IF-MIB::ifType.1 = INTEGER: ethernetCsmacd(6)
IF-MIB::ifType.2 = INTEGER: ethernetCsmacd(6)
IF-MIB::ifType.3 = INTEGER: ethernetCsmacd(6)
IF-MIB::ifType.4 = INTEGER: ethernetCsmacd(6)
IF-MIB::ifMtu.1 = INTEGER: 1500
IF-MIB::ifMtu.2 = INTEGER: 1500
IF-MIB::ifMtu.3 = INTEGER: 1500
IF-MIB::ifMtu.4 = INTEGER: 1500
IF-MIB::ifSpeed.1 = Gauge32: 1000000000
IF-MIB::ifSpeed.2 = Gauge32: 1000000000
IF-MIB::ifSpeed.3 = Gauge32: 0
IF-MIB::ifSpeed.4 = Gauge32: 0
IF-MIB::ifPhysAddress.1 = STRING: 18:a9:5:4e:a7:1c
IF-MIB::ifPhysAddress.2 = STRING: 18:a9:5:4e:a7:1e
IF-MIB::ifPhysAddress.3 = STRING: 18:a9:5:4e:a7:20
IF-MIB::ifPhysAddress.4 = STRING: 18:a9:5:4e:a7:22
IF-MIB::ifAdminStatus.1 = INTEGER: up(1)
IF-MIB::ifAdminStatus.2 = INTEGER: up(1)
IF-MIB::ifAdminStatus.3 = INTEGER: up(1)
IF-MIB::ifAdminStatus.4 = INTEGER: up(1)
IF-MIB::ifOperStatus.1 = INTEGER: up(1)
IF-MIB::ifOperStatus.2 = INTEGER: up(1)
IF-MIB::ifOperStatus.3 = INTEGER: down(2)
IF-MIB::ifOperStatus.4 = INTEGER: down(2)
IF-MIB::ifLastChange.1 = Timeticks: (0) 0:00:00.00
IF-MIB::ifLastChange.2 = Timeticks: (0) 0:00:00.00
IF-MIB::ifLastChange.3 = Timeticks: (0) 0:00:00.00
IF-MIB::ifLastChange.4 = Timeticks: (0) 0:00:00.00
SNMPv2-MIB::snmpInPkts.0 = Counter32: 187
SNMPv2-MIB::snmpInBadVersions.0 = Counter32: 0
SNMPv2-MIB::snmpInBadCommunityNames.0 = Counter32: 0
SNMPv2-MIB::snmpInBadCommunityUses.0 = Counter32: 0
SNMPv2-MIB::snmpInASNParseErrs.0 = Counter32: 0
SNMPv2-MIB::snmpEnableAuthenTraps.0 = INTEGER: disabled(2)
SNMPv2-MIB::snmpSilentDrops.0 = Counter32: 0
SNMPv2-MIB::snmpProxyDrops.0 = Counter32: 0

My thoughts now are simple. SNMP is not enabled in ESXi for the reason that there is not much there to query and you can use the CIM queries that I mentioned in the previous post to look at this instead.

dell · esx · hp · monitoring · snmp

As part of a project I am currently working on I have a requirement to check that my clients’ infrastructure is working to the best of its ability. Whilst we perform regular checks to ensure the sites are running as expected we don’t currently have an easy way to check the health of the ESX hosts that the virtual servers run on. Until now.

I had spent a lot of time trying to “hack” SNMP to be enabled on the ESXi boxes which involved editing the snmp.xml file in the “unsupported” console on the host but after enabling this found that it didnt give me the data I was looking for to run my checks against. Looking a bit further I found a python script which queries the CIM service on the ESX host to find out whether the hardware is working as expected. The script uses the CIM service to check the ESX Health Status and report back to your monitoring platform what the current status of the host is.

Installation is fairly straightforward. The following details are for an Opsview install running on Ubuntu 8.04LTS server but should be easily adaptable to any installation if needs be.

First login to your server as normal and download the latest version of the pywbem module (http://archive.ubuntu.com/ubuntu/pool/universe/p/pywbem/pywbem_0.7.0.orig.tar.gz)

opsview@LON-SVR-MON1:~$ wget http://archive.ubuntu.com/ubuntu/pool/universe/p/pywbem/pywbem_0.7.0.orig.tar.gz

Once you have downloaded the module extract and run the python installer as root

opsview@LON-SVR-MON1:~$ tar -xzf pywbem_0.7.0.orig.tar.gz

opsview@LON-SVR-MON1:~$ cd pywbem-0.7.0/

opsview@LON-SVR-MON1:~/pywbem-0.7.0$ sudo python setup.py install

Next you need to download the check_esx_wbem.py script (http://communities.vmware.com/docs/DOC-7170) and place it in your libexec folder

opsview@LON-SVR-MON1:~/pywbem-0.7.0$ cd /usr/local/nagios/libexec/
opsview@LON-SVR-MON1:/usr/local/nagios/libexec# wget http://communities.vmware.com/servlet/JiveServlet/downloadBody/7170-102-5-4233/check_esx_wbem.py
opsview@LON-SVR-MON1:/usr/local/nagios/libexec# sudo chown nagios:nagios check_esx_wbem.py
opsview@LON-SVR-MON1:/usr/local/nagios/libexec# sudo chmod a+x check_esx_wbem.py

You can test this from the command line using the following command

opsview@LON-SVR-MON1:/usr/local/nagios/libexec# ./check_esx_wbem.py https://10.9.0.65:5989 root Password

In the case above I received the following output but if everything is working as expected the script should return “OK”

WARNING : Power Supply 3 Power Supplies<br>CRITICAL : Power Supply 2 Power Supply 2: Failure detected<br>

Now we have confirmed the script is running we need to add it to Opsview. The first step here is to reload Opsview to pickup the new plugin. Once complete goto Configuration -> Service Checks and Create New Service Check. Setup your check in a similar way to the image below (remember to substitute “root” and “Password” with a valid username and password to login to your ESX host

Save this service check and then apply this to your ESX hosts. If you have multiple ESX hosts that have different username and passwords then you don’t need to create multiple Service Checks as the later versions of Opsview let you specify exceptions when you configure the check for a host

Once you have configured this reload Opsview and wait for Opsview to start checking the ESX server(s). Below is the screenshot from my server with its disconnected PSU

This should now allow you  to keep an eye on your ESX hosts alongside the rest of your network monitoring system.

dell · esx · esxi · health · hp · monitoring · nagios · opsview

Last night I had a few friends over for a quiet night in, a few drinks and a bit of home cooking. Once we had eaten and realised that the collection of games all seem to date back to when I was 10 (think Outburst and Pictionary Junior) we had another look in the cupboard and found a compendium of 36 drinking games. Looking through the games in the compendium realised that most of them were rather too complicated for the amount we had already drunk so we took the two dice that were in the box and use these as a basis to a very simple drinking game.

The rules were incredibly simple, roll the two dice and perform the relevant action that corresponds to the sum of the two dice. The list of actions was as follows:

1. Doesnt exist (there are two dice)
2. Drink 2 digits of your drink
3. Drink 3 digits of your drink
4. Roll Again (3 consecutive roll agains = finish your drink)
5. Choose one other player’s drink and finish it
6. Eat two spoons of coleslaw (it was left over from the meal)
7. Miss a go
8. Repeat the last player’s action
9. Make everyone a drink in 1 minute. If you fail finish everything that y0u have made
10. Finish your own drink
11. All other players drink 3 digits
12. Nominate one other player to finish their drink

After a spate of 9s being rolled in succession and most people being triple parked with their drinks we changed the rules for 9 to sample each players drink.

There were only four of us playing but we did finish off 2 bottles of wine, 2/3 bottle of Vodka, 30+ bottles/cans of beer. It was a good evening and a good game to pass the time.

dice · drinking games · social

I thought that this deserves a special mention.

Backup Exec backs up the DFSr Replicated Folders using the shadow copy components and in the past to perform a restore you were unable to redirect the files to an alternate location. This could cause issues if you wanted to keep both versions of the file as Backup Exec would overwrite the file and then perform an inital replication of that DFSr folder to the other servers in its replication group.

Whilst you could also perform an Authoritative restore of the DFSr folder this has recently caused me even more issues which resulted in support calls to Symantec and Microsoft to follow up on why this happens and what state my DFS is now in as a result of these restores.

During the inital support call to Symantec they advised me that for the first time in Backup Exec you can redirect the files you restore from the Volume Shadow Copy of the DFSr folders. Simply select the server and location in the File Redirection tab in Backup Exec and you will be able to dump the folder structure to whereever you want it and then copy the relevant files back into your DFS structure as you want it.

Backup · Backup Exec · DFS · Microsoft · Replication · Restore · Symantec

Introduction

You really cannot appreciate the need for a solid backup solution until you need to restore that crucial piece of business critical data. Whether it’s a whole server or just one word document it is always important to know that the files are available to be recovered. There is no single solution that works in all scenarios and it is important to select the technologies that meet the needs of the individual site. This article will look at a number of different technologies and try to demonstrate how they can be used in a business environment and help negate the need to use companies like Kroll Ontrack to perform data recovery on hard drives which can be incredibly costly.

Shadow Copy / Previous Version Client

“Shadow Copies for Shared Folders is a new file-storage technology in the Microsoft Windows Server 2003 operating systems. Shadow Copies for Shared Folders provides point-in-time copies of files that are located on a shared network resource, such as a file server. With Shadow Copies for Shared Folders, users can quickly recover deleted or changed files that are stored on the network without administrator assistance, which can increase productivity and reduce administrative costs.” (Shadow Copies for Shared Folders Technical Reference)

This technology is the basis of the Previous Version client and allows recovery of accidently deleted files without having to request tapes or an online restore which may incur further delays in restoring the data. The snapshots are stored on your file server and you should make sure that you have sufficient space to store all your data as well as shadow copies. So that you don’t run out of space on the server a maximum size for the shadow copies is defined and at each snapshot the server will calculate if it can store the next snapshot in the data store without deleting older versions. When it can no longer store new snapshots Shadow Copy will delete the oldest snapshots to make way for the newest changes.

As mentioned this is a nice technology to quickly recover a few files or folders but should NOT be considered a backup solution on its own as you are reliant on your server always being online and having sufficient space to store enough copies of the data that you can restore what you need to. Shadow copy does not allow for hardware failure and should your disk array fail in the server you will lose the data as well as the previous version snapshots.

Tape Backup

Tape backups have been around almost as long as computers have and writing data to a magnetic tape is a tried and tested way of keeping a copy of the data that can be taken off-site to cover the loss of a server. Today backup tapes are able to store up to 1.6TB of data (depending on tape model and compression) on a single cartridge. As a result the tape backup is still widely used today as the backup solution of choice in the workplace as after the initial expenditure of buying the tape drive and software to backup your infrastructure there is little ongoing expense involved in maintaining the tape based backup solution.

The key thing to remember when using a tape based backups is to NOT keep your backup tapes in the same building as the server that you are backing up. You can backup all your data and keep a full year of backups but if they are sitting next to your server and there is a fire you lose both the server and the tapes and are unable to restore the data. It is recommended that once data has been written to tape that the user responsible for changing the tapes removes the tape to a secure location. There are companies, such as Iron Mountain, who offer services to collect tapes on a regular basis and store them in a secure vault. This can give you the peace of mind that you only have the minimum number of tapes on site at any one time.

The number of different backups you keep is completely dependent on how far back you feel you need to recover data. One tape that is overwritten daily is not a safe solution and while it is possible to use a completely new tape for each backup this can quickly become a costly way of backing up data. The most common backup hierarchy is the Grandfather, Father, Son scenario. In this scenario your Son backup would usually be your daily backup and then at the end of each week the Friday/Weekend backup is kept as the Father and at the start of the new week a new set of Son backups is created. At the end of the month the last Father backup is promoted to Grandfather and the process starts again at the beginning of the new month. It is recommended that the Grandfather backups be kept for a set as a reference of the data at that point in time. Over the course of a year using this technology you will need to have 21 tapes to rotate through. (4 tapes for Monday – Thursday, 5 tapes for the Friday/Weekend backups and 12 month end tapes). If you would like to keep two weeks of daily backups you will need a further 4 tapes to cover the second week.

Online Backup

If you have data based across multiple sites or you don’t want to be forced to change tapes on a nightly basis an online backup solution may prove to be a viable solution. In the same way as the tape backup will capture your data on a nightly basis and write it to a magnetic tape the software here will connect to a 3rd party data server and upload the data to be stored here.

Rather than taking a full backup of all the files each night the online backup solutions usually look at taking an initial base backup on site which is integrated into the off-site storage platform and then each night an incremental backup will copy changes since the previous backup to the platform. As a result of this files are stored based on the number of impressions that are pushed to the backup platform i.e a file can be backed up on day 1 but doesn’t change for 2 months at which point the second impression is saved to the backup platform whereas a file that changes daily will write a new impression each time that file is backed up. The number of impressions you want to keep is dependent on the money you are willing to pay for storage.

When planning for an online backup it is important to work out how much data will be changing on a daily basis and needs to be sent across the Internet to the storage platform. If your Internet connection doesn’t have sufficient bandwidth you will not be able to take a full snapshot each night and could end up with gaps in your backups that prevent complete restoration of all the data.

Disaster Recovery Site

If the nature of your business means you cannot afford to be offline whilst your IT infrastructure is restored then a DR site may be something worth considering. If your Infrastructure is severely crippled then you are able to switch core services to another site and your users are able to continue working with minimal disruption.

Microsoft developed the DFS Replication technology in Server 2003 to enable file shares to be replicated between multiple servers in real time. In the case of your primary file server failing you simply need to switch your referral server to your DR site and users will be able to access data through the same file shares and shouldn’t notice the changeover. Replication of databases such as Microsoft Exchange or SQL is not as easy to replicate in the same way as the database files are constantly changing with each access. In these cases 3rd party applications such as DoubleTake or XOSoft (formerly WANSync) can be used to make sure that your databases are replicated in real time to the DR site so they can be switched over as needed. With these scenarios users are able to keep working whilst the core infrastructure is recovered and then any changes made whilst working in the Disaster Recovery scenario can be replicated back to the main offices.

The Disaster Recovery solution is not a cheap solution as you need to pay for a second set of servers to replicate the data to and run in an alternate site such as a data centre however the running costs need to be compared with the cost to the company whilst services are restored.

What should YOU do?

What you do now is a very individual decision based around the needs of your business. There are companies that implement all four different technologies mentioned to provide resilience against there being an issue with any of the other backups however this is a costly solution that is not viable for a number of small companies. For most, implementing either the tape or the online backup along with the Shadow Copy snapshots will provide enough security to restore the data should files be deleted or a server fail.

It should be noted however that the backup to tape or offsite should never be taken for granted and ignored. As part of any backup strategy you should be looking to run test restores from your backup media to ensure that you can recover the data you have backed up.

Backup · disaster recovery · DR · server · tape