vMotion fails with general system error

Not long ago I wanted to vMotion a virtual machine, it failed with a general system error at 80%. First thing that crossed my mind was, it must have a CD/DVD device attached. Wich is usually the reason for failing vMotion. So I editted the virtual machine settings from “client device” to “host device” and back. Checked to make sure there were no VMware tools upgrades that got stuck and tried it again. Still, I got the same error. I vMotionned some other machine off the same host, just to make sure vMotion was still viable. That worked, so vMotion on the cluster was still functional. It had to be something else related to this particular virtual machine. Continue reading

Losing Datastores?

Are you losing Datastores and use VMware Consolidated Backup (VCB). Check if you VCB proxy server is setup correctly. Most importantly, check if automount is disabled on the server. Typically Datastores will disappear fron ESX servers that are not running any Virtual Machines from the Datastore, while ESX servers that are running machines can still see the Datastore. Once you power off those machines, the Datastore will disappear from those ESX servers as well.

Over a year ago, I had this happen on our site. The VCB server was installed by a third party and had been running a while. We just replaced the hardware of some of our ESX 3.5 clusters and for some reason, suddenly, we started losing Datastores. Since the VCB server had been running quite a while, I didn’t think of it straight away. But more and more clues let me to believe this must be the reason for the mystery disappearances. So I checked the VCB server to see if automount was disabled and, sure enough, it wasn’t. I still don’t get why it took so long before this happened. Luckilly there is a way to resolve this. Continue reading

Going for VCAP-DCA/DCD

Being a VCP 3 and 4, the next step for me would be VCAP-DCA followed by VCAP-DCD. I have been dreading this for a while now, but decided I will go for it, despite the fact that I would have to do it all next to my job, kids and wot not.

To get started I downloaded the exam blueprints and, recently, attended the VMware vSphere Troubleshooting course. So the beginnings are there.

Now for the actual studying, there is a lot of reading to be done. Hands on experience with vSphere 4 should not really be a problem, well not this time, anyway, I’ll make sure of that. I had a little bit of a hickup with that part for my vSphere 4 exam. I took that exam without any real hands on experience at all. I only read the book Mastering VMware vSphere 4 (by Scott Lowe) and the vSphere 4.0 Quick Start Guide (By Duncan-Epping, Alan-Renouf, Bernie Baker, Thomas-Bryant, Stuart Radnidge) and, for the most part, relied on my previous experience with VI3. This actually, really, made the exam a lot more difficult than necessary. Although I passed my exam, I certainly won’t be making that mistake again. Continue reading

Virtual machine failed to power on

A while back I got a service request regardig a virtual machine that failed to power on. The error message stated that there was a lock on it’s machine files. Three of them, no less. I had encountered this before and usually it’s a sign that the virtual machine proces is still running on one of the ESX 3.5 hosts in the cluster.

To find the ESX with the running proces for the virtual machine I used a series of commands logged on to the service console of the ESX:

Reclaim unused storage with sVmotion

Last week I was asked to make 2 copies of a virtual machine. The machine in question was a Ubuntu VM with a 250 GB datadisk in thin format. When I checked, the data disk wasn’t so thin anymore. While there was only 21 GB data written to the disk, 240 GB was provisioned. It appeared that, at some point in the history of the machine, someone had written 240 GB data to it and then deleted most of it. I didn’t want to clone the machine (twice) as is, but wanted a lean (thin) datadisk. I thought storage vmotion could provide that, but, as it turns out, there is more to it than just that.

Firstly, when data is deleted, only the pointer to this data, in the filesystem, is removed. The data itself is still present. To really get rid of the data, you have to zero out the disk. This can be done by SDelete by Sysinternals on Windows machines and by using dd on Linux machines. If your disk is thin, you have to remember that zeroing it out will provision the entire disk space, so you have to check if there is enough space left in your datastore before zeroing. Continue reading

VMware tools installer failed to remove old tools

Last week I encountered a Windows Virtual Machine that wouldn’t upgrade its VMware tools after an import from ESX 3.5 to ESXi 4.1 and upgrade of its hardware to HW7. The automatic install failed and an interactive install couldn’t find the required files. When pointing the installer directly to the mounted Tools installer CD, I got this message:

“The file “path to VMware Tools.msi” is not a valid installation package for the product VMware Tools. Try to find the installation package “VMware Tools.msi” in a folder from wich you can install VMware Tools.” Continue reading

Ghost in the machine…

Since upgrading to ESX(i) 4 I have been having some issues with ghosted network devices in server 2008 virtual machines. These ghosted devices appear after deploying a template, cloning a virtual machine, importing a virtual machine and even after upgrading the vmware tools. Server 2008 remembers every device that has been presented to the machine. When these devices are no longer present they are stil there in the device manager as hidden devices.

To show these hidden devices you have to add a environment variable “devmgr_show_nonpresent_devices” with a value of 1. Start the device manager and click on view, show hidden devices. Expand the network adapters tree. The ghosted network adapters will be there as greyed out devices. These devices will have properties, like IP-settings and name. Since they are no longer present you will not be able to change any of these. Continue reading