NSX bug while using Logical Switch as a Destination in the Edge Firewall

While working on a cool VMware NSX project we have discovered a bug within the Edge Firewall when using a “Logical Switch” as a “Destination”.

Lets first start off with making troubleshooting easy and adding the “Rule Tag” and “Log” information to the Firewall view.

Firewall RuleTag

Read the full post »

Creating a single bootable ISO with HP SPP and MSB

This blog post will guide you through the process of creating a single bootable ISO file which consists of the latest HP Service Pack for Proliant (SPP) ISO and an additional HP Maintenance Supplement Bundle (MSB) which is delivered in a ZIP format. Additionally you can add single “Firmware Supplemental Updates” as well.

The HP Service Pack for Proliant is a complete system software and firmware solution that is delivered several times a year, mainly driven by the release of a new server. This SPP is delivered as a bootable ISO and can be used to do offline firmware upgrades. Due to the fact that the SPP is only delivered several times a year it will most likely not contain the most recent updates, that’s why HP lets you also download a Maintenance Supplement Bundle (MSB). The SPP combined with a MSB contains a fully supported set of software.

Read the full post »

Unresponsive/BSOD VM’s on ESXi 5.1 and 5.5

Since a few days I got several customers complaining about unresponsive or blue screening VM’s (both Windows 2008 and 2012) on ESXi5.1 and 5.5 environments. Troubleshooting at the customer site pointed out that the vnetflt.sys driver was causing these issues. This driver is part of the vShield Endpoint components that are installed whenever you a) explicitly installed them or b) when you installed VMware Tools with the “Complete Setup”-option.

Following this VMware KB article is appears that there is a memory leak in the vShield Endpoint and consequently this resolution is described:

This is a known issue affecting VMware Tools 5.1 and can impact ESXi 5.1 and 5.5.
This issue is resolved in:
Currently, there is no resolution in VMware ESXi 5.1.

To resolve this issue when you are using vShield Endpoint Protection in your virtual environment, uninstall and reinstall VMware Tools with the Custom or Complete setup option.

For ESXi 5.5 this is a easy solution but what about ESXi 5.1 environments that are dependent on the vShield Endpoint Components? (like environments running Trend Micro Deep Security or Symantec Endpoint Protection Manager)

There’s another VMware KB article which gives this as a resolution:

This issue is resolved in:

ESXi 5.5 Update 2, available at VMware Downloads. For more information, see the VMware ESXi 5.5 Update 2 Release Notes.

ESXi 5.1 Patch 04, available at VMware Download Patches. For more information, see VMware ESXi 5.1, Patch Release ESXi510-201404001 (2070666).

So be advised on these patches since it’s currently unclear to me why all these customers recently experienced these issues.


Failed to attach filter ‘pxdacf_filter’

During a recent customer visit we had a testing environment available on where some VM’s couldn’t be powered on/vMotionned to some of the ESXi Hosts. The error message:

An error was received from the ESX host while powering on VM xxxxx.
Failed to start the virtual machine.
Module DevicePowerOn power on failed.
Unable to create virtual SCSI device for scsi1:0, ‘/vmfs/volumes/39dfa56f-83350d20/xxxxxx/xxxxxx.vmdk’
Failed to attach filter ‘pxdacf_filter’ to scsi1:0: Not found (195887107).

The error message is similar to the one which VMware is describing in this KB article around vShield Endpoint: The virtual machine power-on operation fails with this error when a virtual machine that was earlier protected by vShield Endpoint is either moved or copied to a host that is not protected by the vShield Endpoint security solution.

However this customer wasn’t using vShield in this test environment and Google didn’t got any hits on the “pxdacf_filter”. Troubleshooting eventually pointed out that some of the ESXi Hosts had Proximal Data AutoCache installed and VM’s that are accelerated by Proximal contain the following lines in their .vmx file:

scsix:x.filters = “pxdacf_filter”

Which obviously caused the VM to be unable to power-on on an ESXi Host without Proximal Data AutoCache installed.

Deep Security 9: Error on call to ‘getaddrinfo’ (DSVA)

During a recent Deep Security implementation we’ve experienced post-deployment issues with the Deep Security Virtual Appliances (DSVA’s). After Deployment of the DSVA’s you will get a “Communications Problem” reported from the Deep Security Manager.


By the way, the Deployment of the DSVA within ESXi 5.5 has got a known issue of not working on the first attempt. Please read this for more information.

The error looks like this:


Read the full post »

Deploying DSVA on ESXi 5.5 fails.

This article describes the scenario where you will experience a timeout from vCenter while deploying the Trend Micro Deep Security Appliance 9.0 (DSVA) on an ESXi 5.5 Host. I got some information from Trend Micro about this issue, which  can simply be resolved by retrying the deployment.

So what happens? The first time you try to Deploy the DSVA (and complete the wizard):


Read the full post »

“Prepare” ESXi 5.5 Host within Deep Security 9.0

During some recent Trend Micro Deep Security 9.0 implementations I’ve come across an issue while “Preparing” the ESXi 5.5 Host for Deep Security. This process is initiated from the Deep Security Manager and should normally install the Trend Micro Filter Driver and add some settings to the ESXi 5.5 Host.

While preparing the ESXi 5.5 Host


You will eventually get the error message: “The installation transaction failed”.

This problem is described in this VMware KB article and a work around is described, however, this work around is not complete.
As VMware describes you need to manually install the Filter Driver and you need to add a Virtual Machine group named “vmservice-trend-pg” to the configuration. This VM Portgroup needs to be added to the same vswitch which has been created by vShield.

Read the full post »

BL460c Gen8: Not detecting SD-Card during ESXi 5.1 Installation

While installing new HP BL460c Gen8’s (Xeon E5-2680 v2) we discovered some strange behavior while setting the HP Power Profile setting to “Maximum Performance”. It appears that this setting causes the ESXi 5.1 Installer to not see the Internal SD-Card as storage device anymore.

Default BIOS Settings give us this, which is good:


Read the full post »

Auto Deploy: ESXi Stateful Install with multiple LUNs connected fails

Recently I have been experiencing some troubles with VMware Host Profiles, Auto Deploy and the stateful install of ESXi 5.1 (Update 1). After the Host Profile gets applied, the system eventually times out with the message: “The request failed because the remote server took too long to respond” as indicated on the screenshot below.

Host Profile Timeout

Read the full post »

Failover difference with SRM4 and SRM5 on HDS VSP

This blog post highlights the technical difference on how vCenter SRM4 and SRM5 do failover with Hitachi’s Virtual Storage Platform (HDS VSP). Credits to Saravanan Mahalingam for delivering this information.


  • SRM4 does not support failback operation
  • SRM4 uses horctakeover command for failover operation. Horctakeover command fails over to remote site and reverses the replication automatically when the volumes are in PAIR status and the remote array is online (in order to make sure the horctakeover command succeeds for TrueCopy sync, fence level of data must be used).
  • When the horctakeover command fails to reverse the replication, the S-VOL is kept in a special state called SSWS. In order to reverse the replication, pairresync -swaps must be executed manually.

 SRM 5

  • SRM5 introduced re-protect and failback operations
  • SRM5 uses pairsplit –RS command for failover operation. Hence the status of S-VOL will always be in SSWS state after the failover
  • Re-protect operation uses the pairresync command to  reverse the replication and make the volumes in PAIR state (S-VOL to the P-VOL and resynchronized the NEW_SVOL based on the NEW_PVOL)
  • Failback/personality swap operation uses the pairresync –swaps command to reverse the replication

So  while SRM4 is reversing the replication automatically, SRM5 needs the “manual” Re-protect option for that. This is important to know in case you need to guarantee replication before VM’s get booted on the Recovery Site (as part of the run book).

For more information on how to implement HDS VSP with vCenter Site Recovery Manager 5 see these resources: