Useful links about Windows Failover Clustering.

Posted on December 15, 2016 by

Free Ebook

http://blogs.technet.com/b/askcore/archive/tags/failover+cluster/default.aspx

 

Understanding the Cluster Debug Log in 2008

http://blogs.technet.com/b/askcore/archive/2010/04/13/understanding-the-cluster-debug-log-in-2008.aspx

 

Troubleshooting Cluster Logs 101 – Why did the resources failover to the other node?

http://blogs.technet.com/b/askcore/archive/2008/02/06/troubleshooting-cluster-logs-101-why-did-the-resources-failover-to-the-other-node.aspx

 

Measuring Disk Latency with Windows Performance Monitor (Perfmon)

http://blogs.technet.com/b/askcore/archive/2012/02/07/measuring-disk-latency-with-windows-performance-monitor-perfmon.aspx

 

Planning Failover Cluster Node Sizing

http://blogs.msdn.com/b/clustering/archive/2014/07/03/10539483.aspx

 

Configuring Windows Failover Cluster Networks

http://blogs.technet.com/b/askcore/archive/2014/02/20/configuring-windows-failover-cluster-networks.aspx

 

Windows Server 2008 R2 Failover Clustering – Best Practice Guide

http://blogs.technet.com/b/aevalshah/archive/2012/05/15/windows-server-2008-r2-failover-clustering-best-practice-guide.aspx

 

Windows Server 2008 R2 Cluster: List of Hotfixes

http://social.technet.microsoft.com/wiki/contents/articles/2008.windows-server-2008-r2-cluster-list-of-hotfixes.aspx

 

What is RHS and what does it do?

http://blogs.msdn.com/b/clustering/archive/2009/06/27/9806160.aspx

 

Resource Hosting Subsystem (RHS) In Windows Server 2008 Failover Clusters

http://blogs.technet.com/b/askcore/archive/2009/11/23/resource-hosting-subsystem-rhs-in-windows-server-2008-failover-clusters.aspx

 

Understanding how Failover Clustering Recovers from Unresponsive Resources

http://blogs.msdn.com/b/clustering/archive/2013/01/24/10388009.aspx

 

978527 The Resource Hosting Subsystem (Rhs.exe) process stops unexpectedly when you start a cluster resource in Windows Server 2008 R2

https://support.microsoft.com/en-us/kb/978527

 

815267 How to enable User Mode Hang Detection on a server cluster in Windows Server 2003 and in Windows 2000 Server SP4

https://support.microsoft.com/kb/815267

 

Decoding Bugcheck 0x0000009E

http://blogs.msdn.com/b/clustering/archive/2013/11/13/10467483.aspx

 

Comparing Hotfixes across Multiple Node Failover Clusters

http://blogs.technet.com/b/askpfeplat/archive/2012/10/15/comparing-hotfixes-across-multiple-node-failover-clusters.aspx

 

Keep your Failover Clustering deployment healthy!

http://blogs.technet.com/b/mspfe/archive/2013/01/11/keep-your-failover-clustering-deployment-healthy.aspx

 

Video – https://mva.microsoft.com/en-us/training-courses/failover-clustering-in-windows-server-2012-r2-8489

Posted in SQL server Failover Cluster, Windows | Tagged , , | Leave a comment

Steps to upgrade the vCSA (PSC\VC ) 6.0 to 6.5.

Posted on November 16, 2016 by

Read the KB 2147548 \ KB 2147686 before upgrading the vshpere environment for the Compatibility considerations and also for the best practices to make sure all the products which is connected to the vsphere will support the 6.5 version.

Update 3/16/2017: As per the VC update 3 release notes “ Upgrading from vCenter Server 6.0 Update 3 to vCenter Server 6.5 is not supported”

Update 8/1/2017: We can now migrate VC6.0 update 3 to VC 6.5 U1

Once ISO is downloaded , open the content inside the folder and we can see the below folder structure and we have several ways to install or upgrade the VCSA 6.5.Basically we have below three types of the installation and Readme will give the clear instruction on the installation methods.

a) The UI based installer
b) The command-line installer
c) Migration from vCenter Server Windows to Appliance

First good factor on 6.5 is no need to have the  Client Integration Plugin and we can see UI Based Installer steps.

Before starting the upgrade, rename the existing  PSC\VC on the vcenter as old because during the upgrade process it will ask  for the target PSC\VC name for the deployment and if it is already on the same VC or ESX then it will show the error as VM name already exists in the vcenter.

Another important thing is we need one temporary IP for migrating the data from the OLD version to the new psc\VC 6.5.

One more item is we need to make sure to remove the DRS from fully automated during the upgrade process.

Upgrading the appliance is the two stage process . First stage involves deploying the new VM to the ESX\VC and the second process is copying the data from the source to destination appliance .

Open Windows Explorer and in the vCenter Server Appliance installation
directory, navigate to the ‘vcsa-ui-installer\win32’ folder.Click on the isntaller.exe

vc6-5

It will open the console to select for the new installation or upgrade.

vc6-51

Next it will show the option and process for upgrade

vc6-53

Accept the license agreement

vc6-54

Next provide the source PSC\VC IP and also select the ESX\VC which the source appliance is running.

wwe

ddd

Next provide the target ESXi\VC in which appliance to be deployed.

untitdled

Next have to provide the Target VM settings to be deployed so like mentioned if we are not renaming the existing VM name in the inventory then it will show the error as already VM name is exists ..

untitdlesd

So make sure the current appliance name is renamed in the inventory.

untieed

Select the datastore

und

Provide the temporary IP to copy the data and make sure it is in same vlan

vland

It will show all the configuration details .

vland_censored

It will go all the below process

untfed

It was in 99% for almost 10 mins while configuring the network and the first phase got completed.

On the second phase it will show the below warning to make sure the appliance is not under fully automated DRS to avoid vmotion of the VM to the other host.

titled

if all the configuration is correct then the PSC installation will be completed.

title1d

We can verify the PSC by login in to the https//PSC:5480 and alsp https://PSC/psc.

Also we can notice the old PSC will be in power-down in Vcenter.

Vcenter Appliance upgrade

Next we can see the vcenter upgrade and we need to follow the same steps as PSC so I will be covering only the second phase of the installation.

Once the first-phase is completed the second phase will start with the pre-upgrade steps .

vcd

Same like PSC it will show the warnings to remove fully automate DRS and also it will highlight to make sure other extension will work with the new upgrade .

war

Next it will show the data size to be copied to the new target server and the size will be depends on our environment.Also we can decide what data we need to copy to the new upgrade target VM.

ddl

It will show the source and target vcenter and its configuration.

unt0d

It will show the warning that source VCenter will be shoutdown.

stitled

upgrade process will be started.

lntitled

We can see the status of copying the data and setting the target VM.

lntif

lndtif

ded

So once the installtion is completed we can take the vcenter using the web and pls note from the Vsphere 6.5 we cant use the desktop C# client to access the vcenter.

 

 

 

 

 

 

 

 

 

 

 

 

 

Posted in Vcenter Appliance, vCSA 6.0, VCSA6.5, VMware | Tagged , , | Leave a comment

vdcrepadmin to find the replication status and design the Platform Services Controller 6.0

Posted on October 31, 2016 by

After attending the VMworld PSC session , I was thinking about to test the VDCREPADMIN tool which helps to find the replication status and to re-desgin the PSC.

Currently we have three PSCs which connects to each other in an in-line fashion, with each PSC installed against the previous PSC, rather than a hub-and-spoke fashion where all of the PSCs would terminate to a central PSC or mesh topology.

 

psc

VDCREPADMIN Showservers is to displays all of the PSCs in a vSphere domain.

Login to the appliance and go to the below path

cd /usr/lib/vmware-vmdir/bin

Run this command to show all PSCs in the vSphere domain:

vdcrepadmin -f showservers -h PSC_FQDN -u administrator -w Administrator_Password

psc1

From the output we can see the PSC Names , Site and Domain.

VDCREPADMIN showpartners is to display the partner PSC.

vdcrepadmin -f showpartners -h PSC_FQDN -u administrator -w Administrator_Password

 

psc2_censored

from the out put we can find the partnership between the PSCs which was installed in an in-line fashion, with each PSC installed against the previous PSC

  • PSC35.* has a replication partnership with PSC236
  • PSC36.* has a replication partnership with both PSC35.* and PSC37.*
  • PSC37.* has a replication partnership with both PSC236

VDCREPADMIN showpartnerstatus is to display the current replication partner of the PSC and also the current replication status between the two nodes.

psc3

Pls note you have to run the showpartnerstatus from each PSC to list the exact partner list and status.

Also from the output we can find the current sync with all the replication partner with the curren update sequence number ( USN ) value and in case of any failure check the log /var/log/vmware/vmdird/vmdird-syslog.log

VDCREPADMIN Createagreement is to create the replication agreements between the PSC with the same vSphere domains and not  between disparate (separate) vSphere domains.

So in our example we are creating the agreement between PSC37 and PSC35 so that in case of PSC36 failure still we have the replication with other partner in the domain.

psc6

Before running the agreement check the current partner.

vdcrepadmin -f showpartners -h PSC_FQDN -u administrator -w Administrator_Password

psc4

Use the following command to create a new replication agreement between PSCs .

vdcrepadmin -f createagreement -2 -h Source_PSC_FQDN -H New_PSC_FQDN_to_Replicate -u administrator -w Administrator_Password

psc5

If we have more number of PSC then plan to have the mesh topology and by using the createagreement we can plan the same. Due to replication time, it may take a few seconds to minutes for a complete mesh topology to be configured.

VDCREPADMIN Removeagreement is to remove the agreement from the replication partner.

First check the current partnership from the specified PSC:

vdcrepadmin -f showpartners -h PSC_FQDN -u administrator -w Administrator_Password 

Use the following command to remove an existing replication agreement between PSCs:

vdcrepadmin -f removeagreement -2 -h Source_PSC_FQDN -H PSC_FQDN_to_Remove_from_Replication -u administrator -w Administrator_Password

psc7

Reference :

KB 2127057

INF8225  – VMworld

 

Posted in Install and Configure VMware vCSA 6.0, Platform Services Controller (PSC ), VC6.0 Appliance Installation Issue, Vcenter Appliance, vCSA 6.0, VMware | Tagged , , , , , | 1 Comment

MLAG issue on CISCO UCS Servers with Arista switch

Posted on September 10, 2016 by

We were trying to configure the MLAG on the Arista switch on each uplink group of the CISCO UCS ( UCSC-C240-M3s) server . Initially ESXi 6.0 was installed and when we trying to enable the LACP on the dVswitch , host got disconnected and also stooped pinging and once we disabled the LACP option it came online. VMware support was involved and asked to install all the firmwares but still we had the same issue.

As a trouble-shooting step we installed the windows OS and once we enable the LACP option in the windows NIC Teaming it started same issue as server got disconnected and also stopped pinging.

Another set of ESX servers with same MLAG configuration on the HP Blade Enclosure on Arista was working fine without any issue .

We involved Cisco for the server and Arista from the switch end for the support , from the Cisco we didnt get any proper respond , finally the issue got escalated to the Tier-1 support in Arista and they found the issue is when the Cisco server is configured for trunk mode, the server is sending LACP frames tagged with a vlan id of 0. Currently, in the version of code we are running on this arista switch platform, it’s dropping these frames as it’s not accepting tagged LACPBPDUs. In addition, when the server is configured as an access port, it is sending the LACP frames tagged with the access vlan specified. Even if there is no default vlan specified, the server is still sending the LACP frames tagged with vlan 0. It seems there is no way to have the Cisco server to send the LACP frames un-tagged like HP blade chassis, tintris which are already on mlag.

Requested the Cisco engineer to see the reason for the server is tagging the LACP frames but didnt get the answer so as per the Arista recommendation we upgraded the  latest code on the switch to Version4.16.7M which fixed the issue.

Reference :

https://eos.arista.com/vmware-esx-5-arista-lacp-guide/

Posted in Arista MLAG, CISCO UCS Servers, ESXi issue, Windows | Tagged , | 1 Comment

Powershell Script to check the DNS Server IPs for the windows 2012 servers in AD.

Posted on August 31, 2016 by

I was asked to get the details in one of our old environment to get the DNS settings for a list of Server 2012 R2 servers in the AD and If the third octet on the primary DNS server setting does not match to the PRD then have to send an email to the operation team.

So I wrote the small script which three below parts to achieve the same.

1. Getting the windows 2012 details from the AD and exporting it to the CSV
2. From the CSV we need to get the DNS settings from the Nic
3. Searching it whether DNS server setting which does not match ‘14’ in third octet then send an email.

Pls download the script from the below link.

Dropbox

 

 

Posted in Powershell, Uncategorized, Windows | Tagged , , , | Leave a comment

Active Director user denied to ESXi SSH login – ESXi 6.0 ( 3620759)

Posted on July 13, 2016 by

After configuring the AD authentication on the ESXi 6.0 as per the KB 2075361 , we were not able to login to the ESX shell using the AD Authentication .

From the /var/log/ auth.log , we have noticed the below error.

pam_access(sshd:auth): access denied for user 

sshd[123225]: [module:pam_lsass]pam_sm_authenticate: failed [error code:40286

As per the KB 2145400 it is mentioned as the known issue and also mentioned the  workaround  but it didnt help so we contacted the VMware support and they fixed the issue by doing the changes below.

1. Run the command
cp /etc/likewise/openldap/ldap.conf /tmp <—- Copy the file ldap.conf to /tmp directory

2. give write permission to /tmp/ldap.conf
chmod +w /tmp/ldap.conf

3. Modify the file /tmp/ldap.conf to set buffer size to 15KB

Replace the line ‘SASL_SECPROPS maxbufsize=40960’ with ‘SASL_SECPROPS maxbufsize=5242880‘ –> 5 MB

buffer

4. Save the file

5. Copy /tmp/ldap.conf to /etc/likewise/openldap/ldap.conf

6. Verify the contents of /etc/likewise/openldap/ldap.conf is modified.

7. restart the likewise daemon lwsmd

/etc/init.d/lwsmd restart

 

 

Posted in ESXi issue, VMware | Tagged | 2 Comments

Trend Micro Deep Security Manager 9.6 ( service pack 1 upgrade ) – Part 5

Posted on July 5, 2016 by

Check my previous blog on DSM database schema changes which have to be done before upgrading the Trend DSM 9.6 to SP1 and also other blogs about DSM 9.5 and 9.6 installation and its functionality.

Download the Trend DSM SP1.

d

Click on the download SP1 file.

led

Accept the Agreement.

2

It will search for the previous version

Select the option to upgrade the existing installation.

3dd

Click next to the installation path.

2led

It will extract the files in the background and start proceeding the installation .

09d

Click finish

Untitlfd

check by login in to the DSM Manager

Untitd

We can check the version of the SP1

Untitlesd

If we have another node, pls do the same steps to upgrade it to the SP1.

Posted in Trend Micro Deep Security | Tagged , , , | 1 Comment