This article is contributed. See the original author and article here.
With AZURE UPDATE COMPLIANCE
& AZURE LOG ANALYTICS
____________________________________________________________________________________________Cory Roberts and Tan Tran
Dear IT Pros,
Recently I and Cory Roberts, Microsoft Sr. CE, worked together on a Customer’s Project, We were upgrading roughly eight thousands Windows 10 devices from multiple versions of Windows 10, (1803, 1809, 1903, 1909) to the current branch 2004. The upgrade deployment has been proceeded with Microsoft Endpoint Manager.
In Endpoint Manager, besides Device Status and End User Update Status, there was not much data provided by Endpoint Monitor or Log. It was hard to troubleshoot the windows 10 feature update process… We decided to go with Azure Update Compliance and Azure Log Analytics Query for monitoring and troubleshooting the Windows Feature Update deployment to match our Customer’s need.
The steps to use Log Analytics for troubleshooting of Endpoint Manager Deployment on Windows 10 Feature Update as follow:
- In Endpoint Manager, create Windows 10 Feature Update Deployment and assign to the related Device Group.
- Create Log Analytics Workspace (if you do not have one).
- Install Update Compliance from Azure Market Place
- Onboarding Update Compliance for Windows 10 devices
- Set Windows 10 Clients to forward telemetry data to Log Analytics Workspace.
- Using Kusto Queries to Monitor and Troubleshoot the Upgrade Process.
____________________________________________________________________________
- In Endpoint Manager, create Windows 10 Feature Update Deployment and assign to the related Device Group
– In Endpoint ManagerDevices,
– Windows 10 Feature Update, Create Profile
– Choose the update to deploy
– Assign to Device Group and create the deployment.
II. Create Log Analytics Workspace (if you do not have one).
- In Azure Portal, search for log analytic workspace
- Creating the Log Analytic Workspace:
- Configure Resource Group and location for Log Analytics Workspace
- Click Create
III. Install Azure Update Compliance from Market Place:
Update Compliance uses Windows 10 diagnostic data for all of its reporting. It collects system data including update deployment progress, Windows Update for Business configuration data, and Delivery Optimization usage data, and then sends this data to a customer-owned Azure Log Analytics workspace to power the experience.
- Update Compliance works only with desktops of Windows 10 Professional, Education, and Enterprise editions. It is not support for Windows Server, Surface Hub, IoT.
- Update Compliance required windows 10 device telemetry at minimum basic level and a Commercial ID, a globally-unique identifier assigned to a specific Solution of Log Analytics workspace.
- After Update Compliance is configured, it could take 48-72 hours before they first appear and continue refreshing its data every 12 hour
- Update Compliance also provide Windows Update Delivery Optimization Status (WUDOAggregratedStatus, WUDOStatus), and Windows Defender Antivirus Threat and Update status (WDAV Threat, WDAVStatus)
To Install Azure Update Compliance
- Go to Azure search and type Update Compliance,
- Choose MarketplaceUpdate Compliance
- Choose the same LogAnalytics Workspace
- Create
Now, the Update Compliance Log will be available for Query search in Log Analytics Workspace as shown here:
To Configure GPO for Update Compliance Clients:
- Go to Computer Configuration>Administrative Templates>Windows ComponentsData Collection and Preview Build
- Choose “Allow Telemetry” and set level of diagnostic to at least basic level
- Choose “Configure the Commercial ID” and copy and paste ID from WaaSUpdateInsight to the GPO setting box
You could view the Commercial ID from the WaaSUpdateInsight as shown:
- Choose “Allow device name to be sent in Windows diagnostic data” and Enabled
IV. Onboarding Update Compliance for Windows 10 Devices.
The Update Compliance Configuration Script is the recommended method of configuring devices to send Telemetry data to Azure Log Analytics Workspace for use with Update Compliance. The script configures device policies via Group Policy, ensures that required services are running, and more.
You can download the script here.
The script is organized into two folders Pilot and Deployment. Both folders have the same key files: ConfigScript.ps1 and RunConfig.bat.
You configure RunConfig.bat according to the directions in the .bat itself, which will then execute ConfigScript.ps1 with the parameters entered to RunConfig.bat.
- The Pilot folder is more verbose and is intended to be use on an initial set of devices and for troubleshooting. Pilot script will collect and output detailed logs
- The Deployment folder is intended to be deployed across an entire device population in a specific environment once devices in that environment have been validated with the Pilot script.
- Configure commercialIDValue in RunConfig.bat to your CommercialID.
- Use a management tool like Configuration Manager or Intune to broadly deploy the script to your entire target population.
Steps to Deploy Update Compliance to Clients:
- Edit PilotRunConfig.bat with Commercial ID of your WaaSInsight Solution and Location for log folder.
- Run the PilotRunConfig.bat and generate errors in report from log folder
- Review log files and correct the problems.
- Edit DeploymentRunConfig.bat with Commercial ID of your WaaSInsight Solution and Location for log folder.
- Run the DeploymentRunConfig.bat. Process may take more than 48hrs for collected data to show up in the Update Compliance Dashboard
V. Set Windows 10 Clients Agent to forward data to Log Analytics Workspace.
Deploy Microsoft Monitoring Agents (MMA) as installation application to all Windows 10 Clients using SCCM.
- Download MMASetup-AMD64.exe and use 7-zip to extract MOMAgent.msi from MMASetup-AMD64.exe
- Create SCCM MMA application using the following command:
msiexec /i MOMAgent.msi ADD_OPINSIGHTS_WORKSPACE=1 OPINSIGHTS_WORKSPACE_ID=Your-WorkspaceID OPINSIGHTS_WORKSPACE_KEY=Your-PrimaryKEY AcceptEndUserLicenseAgreement=1 /q
- Deploy the MOMAgent Application to all Windows 10 SCCM Clients
Configure Log Analytic to collect Windows 10 upgrade logs.
Configure Log Analytics to collect the system event log and application event log together with the Windows update client event logs.
To collect Event Viewer Log for Log Analytics Workspace:
- Log Analytics WorkspaceAdvanced settings
- Choose Data,
- Choose Windows Event Logs
- Type “Application” and click the + button
- Type “System” and click the + button
- Type “Microsoft-Windows-DeviceSetupManager/Admin” and click the + button
- Type “Microsoft-WindowsUpdateClient/Operation” and click the + button
- Save
To collect Windows Upgrade logs for Log Analytics:
There are 4 Windows 10 upgrade phases,
- Downlevel phase: prepare upgrade installer source and destination in Windows 10 OS current version.
- SafeOS phase, WinPE running phase, copying file for setup, prepare disk and file system table if needed, …
- Firstboot phase, Windows system driver installation and reboot.
- Secondboot phase, New version of Windows 10 OS is running, continue installing software applications and drivers.
Depend on the Windows 10 upgrade phases the same upgrade log name could be in different Windows directory locations as shown here, The $WINDOWS~BT path is not working in Log Analytics service:
|
Log file name |
Location |
Suggestions |
|
|
|
|
|
setupact.log |
$Windows.~BTSourcesPanther |
All down-level failures and rollback investigations |
|
setupact.log |
$Windows.~BTSourcesPantherUnattendGC |
OOBE phase rollbacks, 0x4001C, 0x4001D, 0x4001E, 0x4001F |
|
setupact.log |
$Windows.~BTSourcesRollback |
Generic rollbacks, 0xC1900101 |
|
setupact.log |
Windows |
Setup launch failures |
|
setupact.log |
WindowsPanther |
Post-upgrade issues |
|
setuperr.log |
$Windows.~BTSourcesPanther |
Complete error listing |
|
setuperr.log |
$Windows.~BTSourcesPantherUnattendGC |
Complete error listing |
|
setuperr.log |
$Windows.~BTSourcesRollback |
Complete error listing |
|
setuperr.log |
Windows |
Complete error listing |
|
setuperr.log |
WindowsPanther |
Complete error listing |
|
miglog.xml |
WindowsPanther |
Post-upgrade issues |
|
BlueBox.log |
WindowsLogsMosetup |
WSUS and WU down-level failures |
|
setupapi.dev.log |
$Windows.~BTSourcesRollback |
Device install issues |
|
setupapi.dev.log |
C:Windowsinf |
Complete Device install issues |
|
Setupapi.app.log |
C:Windowsinf |
PNP information about operations that install devices and drivers |
|
… |
|
|
- Go to DataCustom log
- Click Add, and “Choose File” button to browse to the log directories specified in the above table.
- Continue add all the logs and path as shown:
- Enter Name of log collection CL, no space allowed in Name.
- Done
You may get permission error, and you would need to “enable inheritance” permission as shown:
VI. Using Kusto Queries to Monitor and Troubleshoot the Upgrade Process.
All the search for upgrade status, update compliance status, Windows update delivery optimizaton information could be done by one tool, the Analytics Workspace Log Query as shown:
To Review Update Log and search for errors:
Run Log Analytics Query to search for update error in windows logs of devices:
- In Azure Portal, Log Analytics Workspace
- Logs, click on + to create new query
- Choose the Custom Logs, double click to insert the related log to Query Windows
- Run Query
Query Custom Logs for All Upgrade errors:
CompleteWindowsSetupLog_CL
| where TimeGenerated <= ago(24m)
| where RawData contains “error“
Query Custom Logs for Upgrade Device Driver Error
PNPDeviceError_CL
| where RawData contains “failure“
Query Custom Logs for Upgrade OOBE and other setup error:
WindowsUpdatePhaseGC_CL
| where RawData contains “error“
| where RawData contains “WimBoot“ or RawData contains “OOBE” or RawData contains “storage“
| project TimeGenerated, Computer, RawData, Type
To Search Event Log for Update Errors:
Query System Event for Update information:
Event
|where TimeGenerated > ago(1d)
| where EventLog contains “system“
| where RenderedDescription contains “Update“
| project TimeGenerated, EventLog, Computer, EventID
To Search Update Compliance Log for Ugrade Errors:
Update Compliance Log provide the pre-built Desktop Analytics queries for all status of update included the following:
- Deployment failures,
- Reboot pending,
- Feature or quality update deferral/pause,
- Update automatic hold by Windows 10 Safeguard (to prevent hardware or software incompatibilities)
In Log Analytics Workspace, there is very useful Update Compliance Logs’ tables related to WaaS and Windows Update Delivery Optimization as shown:
The Update Compliance Logs Query might become our best option in troubleshooting Endpoint Manager Update Feature Deployment.
- Query WaaS for Windows 10 Upgrade with “Not up-to-date” Status:
WaaSUpdateStatus
| where OSFeatureUpdateStatus contains “Not Up-to-date“
| project Computer, LastScan, OSName, OSVersion, FeatureDeferralDays, FeaturePauseState, NeedAttentionStatus
- Query WaaS for Upgrade Deployment with error:
WaaSDeploymentStatus
| where DeploymentErrorCode != “0”
- Query WaaS for Feature Update Deployment not successful, listed by Computer name, Last Scan time, Deployment Status, DetailedStatus…
The DetailedStatus column may show recent feature of Windows 10 2004 with “Safeguard Hold” in the column, Safeguard hold was used to prevent imcompatible device hardware from being upgraded.
WaaSDeploymentStatus
| where DetailedStatus != “UpdateSuccessful“
| where UpdateCategory == “Feature”
| project Computer, LastScan, DeploymentStatus, DeploymentErrorCode, DetailedStatus
- List Feature Update and Quality Update Status of a specific Computer:
WaaSUpdateStatus
| where Computer == “YourComputerName“ and TimeGenerated > ago(30d)
| summarize arg_max(TimeGenerated,OSFeatureUpdateStatus, OSQualityUpdateStatus, NeedAttentionStatus, OSVersion) by Computer
- Query WaaS for Upgrade Deployment with Failed Status and not contain a specific error code:
WaaSDeploymentStatus
| where TimeGenerated > ago(7d)
| where UpdateCategory == ‘Feature’
| where UpdateClassification == ‘Upgrade’
| where DeploymentStatus == ‘Failed‘
| where DeploymentErrorCode notcontains “8007001F”
| where DeploymentError == “N/A”
| where PauseState != “”
Export result to csv file for later investigation of update failure’s root cause.
After we get update error code from query result, then we will need to translate code error to meaningful root cause by using the error reference table from the following link:
https://docs.microsoft.com/en-us/windows/deployment/update/windows-update-error-reference
Continue troubleshooting and testing the deployment of feature updates until we get no error in Log Analytics Query.
You could generate an Intune’s update report-workbook as suggested by Jeff Gilbert blog
References:
Windows update logs file:
https://docs.microsoft.com/en-us/windows/deployment/update/windows-update-logs
Installing Log Analytics Agent for Windows Computers:
https://docs.microsoft.com/en-us/azure/azure-monitor/platform/agent-windows
Update Compliance Setup and Log Analytics Queries:
https://docs.microsoft.com/en-us/windows/deployment/update/update-compliance-get-started
https://docs.microsoft.com/en-us/windows/deployment/update/update-compliance-configuration-manual
https://docs.microsoft.com/en-us/windows/deployment/update/update-compliance-configuration-script
https://docs.microsoft.com/en-us/windows/deployment/update/update-compliance-configuration-manual
https://www.configjon.com/update-compliance-log-analytics-queries/
https://www.jeffgilb.com/update-compliance-with-intune/
Kusto Query tips and examples:
https://docs.microsoft.com/en-us/azure/azure-monitor/log-query/get-started-portal
https://docs.microsoft.com/en-us/azure/azure-monitor/log-query/examples
https://docs.microsoft.com/en-us/azure/data-explorer/kusto/query/samples
I hope the information is useful for Windows Feature Update Troubleshooting.
On my next blogpost, we will re-visit and discuss Update Compliance again.
Cheers!
____________________________________________________________________________________________________
Disclaimer
The sample scripts are not supported under any Microsoft standard support program or service. The sample scripts are provided AS IS without warranty of any kind. Microsoft further disclaims all implied warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The entire risk arising out of the use or performance of the sample scripts and documentation remains with you. In no event shall Microsoft, its authors, or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the sample scripts or documentation, even if Microsoft has been advised of the possibility of such damages.
Brought to you by Dr. Ware, Microsoft Office 365 Silver Partner, Charleston SC.
Recent Comments