Followed the document https://docs.vmware.com/en/VMware-vSphere/6.5/vsphere-esxi-vcenter-server-651-setup-mscs.pdf to configure the windows cluster but it is getting failed with the error
Tried a few VMware KB articles to fix the issue https://kb.vmware.com/s/article/2054897 but all the options got failed and finally the issue got fixed by moving the OS Disk to the RDM LUN.
As per MS normal azcopy depends on the CPU but blob to blob is serverto server APIs so as use AZCOPY_CONCURRENCY_VALUE= more than 1000.
” If you’re copying blobs between storage accounts, consider setting the value of the AZCOPY_CONCURRENCY_VALUE environment variable to a value greater than 1000. You can set this variable high because AzCopy uses server-to-server APIs, so data is copied directly between storage servers and does not use your machine’s processing power. ”
Previously it was taken more than 30 mins to download the data but once we provided AZCOPY_CONCURRENCY_VALUE=2000 and was able to copy from blob to blob in 3.41 minutes.
Azure policy to set the diagnostic Settings for Azure Function to Event Hub so that all the events will be routed to the corresponding event hub.
Azure policy can automate this process and for resources that already exist, a remediation task has to be created.
Below is code that will help to create the Azure Policy to set the Azure Functions diagnostic settings to event-hub.
| { | |
| “properties”: { | |
| “displayName”: “Deploy Diagnostic Settings for Storage Analytics to Event Hub”, | |
| “policyType”: “Custom”, | |
| “mode”: “Indexed”, | |
| “metadata”: { | |
| “createdBy”: “9e1a4c2c-4a14-468c-bad0-0ed38afbb990”, | |
| “createdOn”: “2023-01-17T02:48:32.7901427Z”, | |
| “updatedBy”: null, | |
| “updatedOn”: null, | |
| “category”: “Storage” | |
| }, | |
| “parameters”: { | |
| “eventHubRuleId”: { | |
| “type”: “String”, | |
| “metadata”: { | |
| “displayName”: “Event Hub Authorization Rule Id”, | |
| “description”: “The Event Hub authorization rule Id for Azure Diagnostics. The authorization rule needs to be at Event Hub namespace level. e.g. /subscriptions/{subscription Id}/resourceGroups/{resource group}/providers/Microsoft.EventHub/namespaces/{Event Hub namespace}/authorizationrules/{authorization rule}”, | |
| “strongType”: “Microsoft.EventHub/Namespaces/AuthorizationRules”, | |
| “assignPermissions”: true | |
| } | |
| }, | |
| “eventHubName”: { | |
| “type”: “String”, | |
| “metadata”: { | |
| “displayName”: “Event Hub name”, | |
| “assignPermissions”: true | |
| } | |
| }, | |
| “eventHubLocation”: { | |
| “type”: “String”, | |
| “metadata”: { | |
| “displayName”: “Event Hub Location”, | |
| “description”: “The location the Event Hub resides in. Only Logic Apps in this location will be linked to this Event Hub.”, | |
| “strongType”: “location” | |
| }, | |
| “defaultValue”: “” | |
| }, | |
| “servicesToDeploy”: { | |
| “type”: “Array”, | |
| “metadata”: { | |
| “displayName”: “Storage services to deploy”, | |
| “description”: “List of Storage services to deploy” | |
| }, | |
| “allowedValues”: [ | |
| “storageAccounts”, | |
| “blobServices”, | |
| “fileServices”, | |
| “tableServices”, | |
| “queueServices” | |
| ], | |
| “defaultValue”: [ | |
| “storageAccounts”, | |
| “blobServices”, | |
| “fileServices”, | |
| “tableServices”, | |
| “queueServices” | |
| ] | |
| }, | |
| “diagnosticsSettingNameToUse”: { | |
| “type”: “String”, | |
| “metadata”: { | |
| “displayName”: “Setting name”, | |
| “description”: “Name of the diagnostic settings.” | |
| }, | |
| “defaultValue”: “storageAccountsDiagnosticsLogsToEventHub” | |
| }, | |
| “effect”: { | |
| “type”: “String”, | |
| “metadata”: { | |
| “displayName”: “Effect”, | |
| “description”: “Enable or disable the execution of the policy” | |
| }, | |
| “allowedValues”: [ | |
| “DeployIfNotExists”, | |
| “Disabled” | |
| ], | |
| “defaultValue”: “DeployIfNotExists” | |
| }, | |
| “StorageDelete”: { | |
| “type”: “String”, | |
| “metadata”: { | |
| “displayName”: “StorageDelete – Enabled”, | |
| “description”: “Whether to stream StorageDelete logs to the Log Analytics workspace – True or False” | |
| }, | |
| “allowedValues”: [ | |
| “True”, | |
| “False” | |
| ], | |
| “defaultValue”: “True” | |
| }, | |
| “StorageWrite”: { | |
| “type”: “String”, | |
| “metadata”: { | |
| “displayName”: “StorageWrite – Enabled”, | |
| “description”: “Whether to stream StorageWrite logs to the Log Analytics workspace – True or False” | |
| }, | |
| “allowedValues”: [ | |
| “True”, | |
| “False” | |
| ], | |
| “defaultValue”: “True” | |
| }, | |
| “StorageRead”: { | |
| “type”: “String”, | |
| “metadata”: { | |
| “displayName”: “StorageRead – Enabled”, | |
| “description”: “Whether to stream StorageRead logs to the Log Analytics workspace – True or False” | |
| }, | |
| “allowedValues”: [ | |
| “True”, | |
| “False” | |
| ], | |
| “defaultValue”: “True” | |
| }, | |
| “Transaction”: { | |
| “type”: “String”, | |
| “metadata”: { | |
| “displayName”: “Transaction – Enabled”, | |
| “description”: “Whether to stream Transaction logs to the Log Analytics workspace – True or False” | |
| }, | |
| “allowedValues”: [ | |
| “True”, | |
| “False” | |
| ], | |
| “defaultValue”: “True” | |
| } | |
| }, | |
| “policyRule”: { | |
| “if”: { | |
| “allOf”: [ | |
| { | |
| “field”: “type”, | |
| “equals”: “Microsoft.Storage/storageAccounts” | |
| }, | |
| { | |
| “anyOf”: [ | |
| { | |
| “value”: “[parameters(‘eventHubLocation’)]”, | |
| “equals”: “” | |
| }, | |
| { | |
| “field”: “location”, | |
| “equals”: “[parameters(‘eventHubLocation’)]” | |
| } | |
| ] | |
| } | |
| ] | |
| }, | |
| “then”: { | |
| “effect”: “[parameters(‘effect’)]”, | |
| “details”: { | |
| “type”: “Microsoft.Insights/diagnosticSettings”, | |
| “roleDefinitionIds”: [ | |
| “/providers/Microsoft.Authorization/roleDefinitions/749f88d5-cbae-40b8-bcfc-e573ddc772fa”, | |
| “/providers/Microsoft.Authorization/roleDefinitions/f526a384-b230-433a-b45c-95f59c4a2dec”, | |
| “/providers/Microsoft.Authorization/roleDefinitions/b24988ac-6180-42a0-ab88-20f7382dd24c” | |
| ], | |
| “existenceCondition”: { | |
| “allOf”: [ | |
| { | |
| “count”: { | |
| “field”: “Microsoft.Insights/diagnosticSettings/metrics[*]”, | |
| “where”: { | |
| “allOf”: [ | |
| { | |
| “field”: “Microsoft.Insights/diagnosticSettings/metrics[*].category”, | |
| “equals”: “Transaction” | |
| }, | |
| { | |
| “field”: “Microsoft.Insights/diagnosticSettings/metrics[*].enabled”, | |
| “equals”: “True” | |
| } | |
| ] | |
| } | |
| }, | |
| “greater”: 0 | |
| }, | |
| { | |
| “field”: “Microsoft.Insights/diagnosticSettings/logs.enabled”, | |
| “contains”: “true” | |
| }, | |
| { | |
| “field”: “Microsoft.Insights/diagnosticSettings/eventHubAuthorizationRuleId”, | |
| “equals”: “[parameters(‘eventHubRuleId’)]” | |
| } | |
| ] | |
| }, | |
| “deployment”: { | |
| “properties”: { | |
| “mode”: “incremental”, | |
| “template”: { | |
| “$schema”: “http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#”, | |
| “contentVersion”: “1.0.0.0”, | |
| “parameters”: { | |
| “servicesToDeploy”: { | |
| “type”: “array” | |
| }, | |
| “diagnosticsSettingNameToUse”: { | |
| “type”: “string” | |
| }, | |
| “resourceName”: { | |
| “type”: “string” | |
| }, | |
| “eventHubRuleId”: { | |
| “type”: “string” | |
| }, | |
| “eventHubName”: { | |
| “type”: “string” | |
| }, | |
| “location”: { | |
| “type”: “string” | |
| }, | |
| “Transaction”: { | |
| “type”: “string” | |
| }, | |
| “StorageRead”: { | |
| “type”: “string” | |
| }, | |
| “StorageWrite”: { | |
| “type”: “string” | |
| }, | |
| “StorageDelete”: { | |
| “type”: “string” | |
| } | |
| }, | |
| “variables”: {}, | |
| “resources”: [ | |
| { | |
| “condition”: “[contains(parameters(‘servicesToDeploy’), ‘blobServices’)]”, | |
| “type”: “Microsoft.Storage/storageAccounts/blobServices/providers/diagnosticSettings”, | |
| “apiVersion”: “2017-05-01-preview”, | |
| “name”: “[concat(parameters(‘resourceName’), ‘/default/’, ‘Microsoft.Insights/’, parameters(‘diagnosticsSettingNameToUse’))]”, | |
| “location”: “[parameters(‘location’)]”, | |
| “dependsOn”: [], | |
| “properties”: { | |
| “eventHubAuthorizationRuleId”: “[parameters(‘eventHubRuleId’)]”, | |
| “eventHubName”: “[parameters(‘eventHubName’)]”, | |
| “metrics”: [ | |
| { | |
| “category”: “Transaction”, | |
| “enabled”: “[parameters(‘Transaction’)]”, | |
| “retentionPolicy”: { | |
| “days”: 0, | |
| “enabled”: false | |
| }, | |
| “timeGrain”: null | |
| } | |
| ], | |
| “logs”: [ | |
| { | |
| “category”: “StorageRead”, | |
| “enabled”: “[parameters(‘StorageRead’)]” | |
| }, | |
| { | |
| “category”: “StorageWrite”, | |
| “enabled”: “[parameters(‘StorageWrite’)]” | |
| }, | |
| { | |
| “category”: “StorageDelete”, | |
| “enabled”: “[parameters(‘StorageDelete’)]” | |
| } | |
| ] | |
| } | |
| }, | |
| { | |
| “condition”: “[contains(parameters(‘servicesToDeploy’), ‘fileServices’)]”, | |
| “type”: “Microsoft.Storage/storageAccounts/fileServices/providers/diagnosticSettings”, | |
| “apiVersion”: “2017-05-01-preview”, | |
| “name”: “[concat(parameters(‘resourceName’), ‘/default/’, ‘Microsoft.Insights/’, parameters(‘diagnosticsSettingNameToUse’))]”, | |
| “location”: “[parameters(‘location’)]”, | |
| “dependsOn”: [], | |
| “properties”: { | |
| “eventHubAuthorizationRuleId”: “[parameters(‘eventHubRuleId’)]”, | |
| “eventHubName”: “[parameters(‘eventHubName’)]”, | |
| “metrics”: [ | |
| { | |
| “category”: “Transaction”, | |
| “enabled”: “[parameters(‘Transaction’)]”, | |
| “retentionPolicy”: { | |
| “days”: 0, | |
| “enabled”: false | |
| }, | |
| “timeGrain”: null | |
| } | |
| ], | |
| “logs”: [ | |
| { | |
| “category”: “StorageRead”, | |
| “enabled”: “[parameters(‘StorageRead’)]” | |
| }, | |
| { | |
| “category”: “StorageWrite”, | |
| “enabled”: “[parameters(‘StorageWrite’)]” | |
| }, | |
| { | |
| “category”: “StorageDelete”, | |
| “enabled”: “[parameters(‘StorageDelete’)]” | |
| } | |
| ] | |
| } | |
| }, | |
| { | |
| “condition”: “[contains(parameters(‘servicesToDeploy’), ‘tableServices’)]”, | |
| “type”: “Microsoft.Storage/storageAccounts/tableServices/providers/diagnosticSettings”, | |
| “apiVersion”: “2017-05-01-preview”, | |
| “name”: “[concat(parameters(‘resourceName’), ‘/default/’, ‘Microsoft.Insights/’, parameters(‘diagnosticsSettingNameToUse’))]”, | |
| “location”: “[parameters(‘location’)]”, | |
| “dependsOn”: [], | |
| “properties”: { | |
| “eventHubAuthorizationRuleId”: “[parameters(‘eventHubRuleId’)]”, | |
| “eventHubName”: “[parameters(‘eventHubName’)]”, | |
| “metrics”: [ | |
| { | |
| “category”: “Transaction”, | |
| “enabled”: “[parameters(‘Transaction’)]”, | |
| “retentionPolicy”: { | |
| “days”: 0, | |
| “enabled”: false | |
| }, | |
| “timeGrain”: null | |
| } | |
| ], | |
| “logs”: [ | |
| { | |
| “category”: “StorageRead”, | |
| “enabled”: “[parameters(‘StorageRead’)]” | |
| }, | |
| { | |
| “category”: “StorageWrite”, | |
| “enabled”: “[parameters(‘StorageWrite’)]” | |
| }, | |
| { | |
| “category”: “StorageDelete”, | |
| “enabled”: “[parameters(‘StorageDelete’)]” | |
| } | |
| ] | |
| } | |
| }, | |
| { | |
| “condition”: “[contains(parameters(‘servicesToDeploy’), ‘queueServices’)]”, | |
| “type”: “Microsoft.Storage/storageAccounts/queueServices/providers/diagnosticSettings”, | |
| “apiVersion”: “2017-05-01-preview”, | |
| “name”: “[concat(parameters(‘resourceName’), ‘/default/’, ‘Microsoft.Insights/’, parameters(‘diagnosticsSettingNameToUse’))]”, | |
| “location”: “[parameters(‘location’)]”, | |
| “dependsOn”: [], | |
| “properties”: { | |
| “eventHubAuthorizationRuleId”: “[parameters(‘eventHubRuleId’)]”, | |
| “eventHubName”: “[parameters(‘eventHubName’)]”, | |
| “metrics”: [ | |
| { | |
| “category”: “Transaction”, | |
| “enabled”: “[parameters(‘Transaction’)]”, | |
| “retentionPolicy”: { | |
| “days”: 0, | |
| “enabled”: false | |
| }, | |
| “timeGrain”: null | |
| } | |
| ], | |
| “logs”: [ | |
| { | |
| “category”: “StorageRead”, | |
| “enabled”: “[parameters(‘StorageRead’)]” | |
| }, | |
| { | |
| “category”: “StorageWrite”, | |
| “enabled”: “[parameters(‘StorageWrite’)]” | |
| }, | |
| { | |
| “category”: “StorageDelete”, | |
| “enabled”: “[parameters(‘StorageDelete’)]” | |
| } | |
| ] | |
| } | |
| }, | |
| { | |
| “condition”: “[contains(parameters(‘servicesToDeploy’), ‘storageAccounts’)]”, | |
| “type”: “Microsoft.Storage/storageAccounts/providers/diagnosticSettings”, | |
| “apiVersion”: “2017-05-01-preview”, | |
| “name”: “[concat(parameters(‘resourceName’), ‘/’, ‘Microsoft.Insights/’, parameters(‘diagnosticsSettingNameToUse’))]”, | |
| “location”: “[parameters(‘location’)]”, | |
| “dependsOn”: [], | |
| “properties”: { | |
| “eventHubAuthorizationRuleId”: “[parameters(‘eventHubRuleId’)]”, | |
| “eventHubName”: “[parameters(‘eventHubName’)]”, | |
| “metrics”: [ | |
| { | |
| “category”: “Transaction”, | |
| “enabled”: “[parameters(‘Transaction’)]”, | |
| “retentionPolicy”: { | |
| “days”: 0, | |
| “enabled”: false | |
| }, | |
| “timeGrain”: null | |
| } | |
| ] | |
| } | |
| } | |
| ], | |
| “outputs”: {} | |
| }, | |
| “parameters”: { | |
| “diagnosticsSettingNameToUse”: { | |
| “value”: “[parameters(‘diagnosticsSettingNameToUse’)]” | |
| }, | |
| “eventHubRuleId”: { | |
| “value”: “[parameters(‘eventHubRuleId’)]” | |
| }, | |
| “eventHubName”: { | |
| “value”: “[parameters(‘eventHubName’)]” | |
| }, | |
| “location”: { | |
| “value”: “[field(‘location’)]” | |
| }, | |
| “resourceName”: { | |
| “value”: “[field(‘name’)]” | |
| }, | |
| “Transaction”: { | |
| “value”: “[parameters(‘Transaction’)]” | |
| }, | |
| “StorageDelete”: { | |
| “value”: “[parameters(‘StorageDelete’)]” | |
| }, | |
| “StorageWrite”: { | |
| “value”: “[parameters(‘StorageWrite’)]” | |
| }, | |
| “StorageRead”: { | |
| “value”: “[parameters(‘StorageRead’)]” | |
| }, | |
| “servicesToDeploy”: { | |
| “value”: “[parameters(‘servicesToDeploy’)]” | |
| } | |
| } | |
| } | |
| } | |
| } | |
| } | |
| } | |
| }, | |
| “id”: “/subscriptions/6cae13a3-4be0-48e0-9466-d9f2f0f33bc9/providers/Microsoft.Authorization/policyDefinitions/9ecfc4b5-5444-4c6e-832e-c06ac3ef2ecc”, | |
| “type”: “Microsoft.Authorization/policyDefinitions”, | |
| “name”: “9ecfc4b5-5444-4c6e-832e-c06ac3ef2ecc”, | |
| “systemData”: { | |
| “createdBy”: “ganesh.sekarbabu@autodesk.com”, | |
| “createdByType”: “User”, | |
| “createdAt”: “2023-01-17T02:48:32.7728818Z”, | |
| “lastModifiedBy”: “ganesh.sekarbabu@autodesk.com”, | |
| “lastModifiedByType”: “User”, | |
| “lastModifiedAt”: “2023-01-17T02:48:32.7728818Z” | |
| } | |
| } |
The issue got started after upgrading the vCenter Server 7.0 Update 3h | 13 SEP 2022 | ISO Build 20395099. We have noticed the CPU of the VC went very high and couldn’t login to the VC. We tried a few options like restarting the services and rebooting the VC but nothing helped to fix the issue.
Later we identified the VPXA log rotation was very fast, every 5 -7 mins new logs are generated so raised the ticket with VMWare and it is confirmed that it is a known issue on this patch and recommended upgrading the VC to the latest.
After upgrading the VC to the vCenter Server 7.0 Update 3i | 08 DEC 2022 | ISO Build 20845200 verified the CPU and it is less than 10%.
We didn’t find any resolved issue in the patch release notes but the latest patch helped to fix the issue.
Note: VMware has released another new patch so pls plan to go with whatever latest available patch.
Already mentioned in my previous blog here about the NFS 4 slowness issue compared to the NFS3 and we captured the packets and worked with the Netapp, below are the findings of our test when we tried ls -lahR | wc -l
We decided to test it on the same single datastore by mounting it as NFS 3 and capturing the data, once the test is done then unmount it and mount it as NFS 4 so on the same datastore with the same size and data below are the result.
Based on the result, VMware will be working on the option to enable the Lookup cache and mostly it will be available in future ESXi patches.
In our organization, we have restrictions to use all the services in azure and have to work with the different teams to enable the resource types in policy for the resources.
Recently I was working to add the non-azure VM in Azure ARC and it was failing with an error.
“FATAL RequestCorrelationId:281e537a-90cfa3a4003 Message: Resource ‘12345’ was disallowed by policy. ” while troubleshooting found this link which helped to find the policy and it is very easy to search
https://www.azadvertizer.net/azpolicyadvertizer_all.html#%7B%7D
For EX to find the appropriate policy, we can search Azure ARC and it will list all the categories.
We can see for the guest configuration “Microsoft.HybridCompute/machines” resource types.
Recently we have noticed in a few datastore the search operation is taking more time than expected and in our testing, it was identified that compare to NFSv4, NFSv3 results are better.
Our testing is to first list files from the host shell and we mounted the same storage as nfsv4 and nfsv3 to the same host. We ran this command from the host shell against both storages. time ls -lahR | wc -l . nfsv4 takes 1 minute and 30 seconds to finish running this command. When it is nfsv3, it only takes 15 seconds.
Then tried using Powershell. Use the below way to search files.
$searchspec = New-Object VMware.Vim.HostDatastoreBrowserSearchSpec
$searchSpec.matchpattern = “*.vmx”
$taskMoRef = $dsBrowser.SearchDatastoreSubFolders_Task($datastorePath2, $searchSpec)
The data transfer rate is pretty much the same but the slowness issue is at the list/search file for nfs4.
Based on the results we involved the vendors to investigate the issue. NetApp conducted a thorough investigation and determined there were no performance issues, but VMware acknowledged there was a problem with NFS v4.1. Below was the reply from the VMware.
“Based on the analysis, Engineering team has identified that the issue related to slow search on NFSv4.1 is caused because the NFSv4.1 does not support “Directory Name Lookup Cache(DNLC) yet. However, for NFSv3, most of the LOOKUP calls are served from cache, which avoids sending a LOOKUP instruction to the NFS server. The VMware engineering is working to add this feature for NFSv4.1 however we do not have a version confirmation where this is expected to be included.”
We applied the latest patches but the issue still exists and hopefully, in the future patch it will get fixed.
Update: OCT2022
For more details check the blog.
Azure AD has been configured and we are able to login to the Azure VM from another Azure VM using the AD credentials but it is getting failed when we try to connect using our local laptop.
One of the prerequisites is to make sure the local laptop should show AzureAdJoined : YES but still having issues and the error it failed is ” The logon attempt failed”.
dsregcmd /status it was showing the AzureAdJoined : YES.
After a few searches, identified the issue because the local GPO applied to the laptop.
Specifically, this is called out in the doc for AAD Login to Windows VMs here: https://docs.microsoft.com/en-us/azure/active-directory/devices/howto-vm-sign-in-azure-ad-windows#unauthorized-client
Here’s the doc for that particular setting: https://docs.microsoft.com/en-us/windows/security/threat-protection/security-policy-settings/network-security-allow-pku2u-authentication-requests-to-this-computer-to-use-online-identities
Reference:
Techbrainblog 