Monitoring Storage Replication – Part 1

by | Feb 17, 2023 | Technology | 0 comments

This article is contributed. See the original author and article here.

Introduction


 


Hi folks! My name is Felipe Binotto, Cloud Solution Architect, based in Australia.


We all know how frustrating it can be to receive a call about a storage account not replicating or being unable to fail over. To help prevent this from happening, I am going to show you how to monitor the replication of your storage accounts. Keep in mind that replication logs are not available as part of the storage account’s diagnostic settings.


 


Pre-Requisites


 


Before we begin, please ensure that you have the following prerequisites in place:



  • Azure Subscription

  • Automation Account

  • Log Analytics workspace


 


High-Level Steps


 


The process for monitoring storage account replication can be broken down into several high-level steps, which we will go through in the following order:


 



  1. Clone the repository that contains the runbook.

  2. Create a user-assigned managed identity.

  3. Provide the identity with the necessary access.

  4. Assign the identity to the Automation Account.

  5. Import the runbook to the Automation Account.

  6. Provide values for the Runbook variables.

  7. Create a new Automation Account variable.

  8. Run the runbook to retrieve the storage account replication information.


 


Getting Started


 


Clone the repo by running the following command:


 


 

git clone https://github.com/fbinotto/storagereplication.git

 


 


Create a new User Managed Assigned Identity.


 


 

$id = (New-AzUserAssignedIdentity -Name storagereplication `
-ResourceGroupName REPLACE_WITH_YOUR_RG –Location australiaeast)

 


 


Assign the identity Storage Account Contributor rights in your subscription(s) so it can retrieve the replication information from Storage Accounts.


 


 

New-AzRoleAssignment -ObjectId $id.PrincipalId `
-RoleDefinitionName 'Storage Account Contributor' `
-Scope /subscriptions/

New-AzRoleAssignment -ObjectId $id.PrincipalId `
-RoleDefinitionName 'Log Analytics Contributor' `
-Scope /subscriptions/

 


 


 


Now assign the identity to the Automation Account.


 


 

# Get the automation account
$automationAccount = Get-AzAutomationAccount -ResourceGroupName REPLACE_WITH_YOUR_RG -Name REPLACE_WITH_YOUR_AA

# Assign the identity to the automation account
$automationAccount | Set-AzAutomationAccount -Identity $id

 


 


Import the runbook to your automation account. Make sure you run the next command from the folder which was cloned.


 


 

Import-AzAutomationRunbook -Path ".storagereplication.ps1" `
-Name StorageReplication –Published:$true `
-ResourceGroupName REPLACE_WITH_YOUR_RG `
-AutomationAccountName  REPLACE_WITH_YOUR_AA -Type PowerShell

 


 


Open the script in VS Code or you can edit straight in your automation account. Now I will highlight some of the important sections, so you have a clear understanding of what is going on.


 


The script is used to collect and send replication logs for Azure Storage Accounts to a Log Analytics workspace. By using this script, you can monitor the replication of your Storage Accounts, so that you can be alerted if there are any issues and act before it becomes a problem.


 


The script starts by setting some variables, including the ID of the Log Analytics workspace, the primary key for authentication, and the name of the record type that will be created.


 


The primary key is retrieved from an Automation Account variable, so we don’t expose it in clear text. Run the following command to create the variable.


 


 

# Create the encrypted variable
New-AzAutomationEncryptedVariable -AutomationAccountName REPLACE_WITH_YOUR_AA -ResourceGroupName REPLACE_WITH_YOUR_RG -Name SharedKey -Value REPLACE_WITH_YOUR_LOG_ANALYTICS_PRIMARY_KEY

 


 


The script then defines two functions: Build-Signature and Post-LogAnalyticsData.


 


The Build-Signature function creates an authorization signature that will be used to authenticate the request to the Log Analytics API. The function takes in several parameters, including the ID of the Log Analytics workspace, the primary key, the date, the content length, the method, the content type, and the resource.


 


The Post-LogAnalyticsData function creates and sends the request to the Log Analytics API. This function takes in the ID of the Log Analytics workspace, the primary key, the body of the request (which contains the replication logs), the log type, and the timestamp field.


The script also includes a line of code (Disable-AzContextAutosave) that ensures that the runbook does not inherit an AzContext, which can cause issues when running the script.


 


Finally, the script calls the Post-LogAnalyticsData function, sending the replication logs to the Log Analytics workspace.


At this point you can run the Runbook. Once the logs have been sent, you can create Azure Alerts based on KQL queries to notify you of any issues with the replication.


 


For example, the following code would return Storage Accounts which have not replicated in the last 8 hours.


 


 

StorageReplicationHealth_CL
| where todatetime(Storage_LastSyncTime_s) < ago(8h)

 


 


In Part 2 of this post, I will demonstrate how you can leverage Logic Apps to send out customized emails when your Storage Account is not replicating.


 


Conclusion


In conclusion, monitoring the replication of your Azure Storage Accounts is crucial to ensure the availability and reliability of your data. In this blog post, we have shown you how to set up monitoring for your Storage Accounts using Log Analytics and Azure Automation. By following the steps outlined in this post and using the provided script, you will be able to monitor the replication status of your Storage Accounts and receive alerts if there are any issues. This will allow you to act quickly and prevent any disruptions to your services. With this solution in place, you can have peace of mind knowing that your data is safe and available.


 


I hope this was informative to you and thanks for reading!


 


 


Disclaimer


The sample scripts are not supported under any Microsoft standard support program or service. The sample scripts are provided AS IS without warranty of any kind. Microsoft further disclaims all implied warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The entire risk arising out of the use or performance of the sample scripts and documentation remains with you. In no event shall Microsoft, its authors, or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the sample scripts or documentation, even if Microsoft has been advised of the possibility of such damages.

Brought to you by Dr. Ware, Microsoft Office 365 Silver Partner, Charleston SC.

Submit a Comment