Introduction to Az.Tool.Installer

by Scott Muniz | Sep 21, 2020 | Uncategorized

This article is contributed. See the original author and article here.

I. Background

Azure PowerShell is a faster iterative product. We expect our customers to update their locally installed versions constantly to stay up to date. The most common way to update it is to use PowerShellGet, a tool built-in to PowerShell for module management.

Usually, you run “Install-Module -Name Az” to install the latest versions of Azure PowerShell modules. When you run “Install-Module -Name $moduleName” to install a specific Az module, you’ll find both the target module and Az.Accounts are installed.

Figure –1 Dependencies among Az Modules

From the behaviors, you will find Az is a bundle of several modules, which depends on all sub-modules of Az rather than a simple single module. When running Install-Module, PowerShellGet fetches Az’s dependencies from the PowerShell gallery and installs the module itself with all the modules it depends on. Meanwhile, each sub-module of Az, for example, Az.Compute, also depends on Az.Accounts. That’s why Az.Accounts is always installed regardless of what Azure PowerShell module is installed. Figure 1, as shown above, illustrates the relationship between the Azure PowerShell modules.

II. Problems

1. Uninstaller’s Negligence

You might expect uninstallation to be a reverse process of installation. Unfortunately, it is not. “Uninstall-Module -Name Az” only removes the Az module without removing the sub-modules. A similar installation and uninstallation process happens to AzureRM. That’s why many people who are migrating from AzureRM to Az have difficulty removing the AzureRM modules.

The package’s dependency information is stored in the original NuGet package but discarded when unpacking the package downloaded during the installation process. When uninstalling the Az module, PowerShellGet doesn’t know the relationship between Az and its sub-modules. Therefore, it only uninstalls the Az module and not the sub-modules.

2. Forgotten Preview Modules

Azure PowerShell often adds new modules. When a new module first onboards, it is in preview mode. This allows for breaking changes in the subsequent versions. At this stage, its version starts with zero, such as 0.1.0. We have released 104 Az sub-modules, 45 of them are in preview mode. When a module goes stable, we upgrade its version to start with a non-zero value, such as 1.0.0, which means it is a general release. After generally released, a module can still release preview versions to include new unstable features. Almost every module contains a preview version before or now. For convenience, we refer to both as being in preview mode and the modules’ preview versions as preview modules.

Most users are unaware of preview modules because they are released independently, not part of the Az module bundle. Sometimes it is difficult to find preview modules, especially the modules without general release versions unless you know their exact names. Since they are not in the Az module bundle, you can only install them individually.

3. Risky Update

If you have several scripts based on Az modules, you need to check whether the upcoming update contains a breaking change (the upgrade of the major version number). “Update-Module” helps you to do the update, but it does everything silently. You must perform the verification manually before running it.

III. Our solution: Az.Tool.Installer

To solve the problems mentioned above, we are launching a new tool, Az.Tool.Installer. It’s based on PowerShellGet and solves the issues we’ve found with it. The first version has three cmdlets which cover installation, uninstallation, and updating.

Compared to PowerShellGet, Az.Tool.Installer perceives the relationship between Az and sub-modules and can manipulate them as a bundle. It collects the information via “Find-Module” from PowerShellGet, which returns the dependency relationship between modules. We also use the cmdlet to retrieve the preview modules.

As Az.Tool.Installer performs the search before the writing operation which causes its cmdlets to take more time to run than similar ones in PowerShellGet. A local cache might shorten the running time though. However, we’re not currently using local caching because it’s easily invalidated since Azure PowerShell changes rapidly.

1. Install-AzModule

Table-1 The Syntax of Install-AzModule

1.1 Clear the Environment and Install Az with the Specific Version

• Install-AzModule -RequiredVersion 4.5.0 -Repository PSGallery –RemovePrevious – RemoveAzureRm [-Name Storage,Network]

You can uninstall everything related to Azure PowerShell and AzureRM on your local machine and install all the modules from a target Az release easily with Install-AzModule. If “-RemovePrevious” is present, the cmdlet removes all the modules that the previous version installed. If “-RemoveAzureRm”, it removes all the modules from AzureRM.

You can install modules with the Az release version by specifying “-RequiredVersion” or with module names by specifying “-Name”.

Note: Install-AzModule only installs Az sub-modules without the Az module itself.

Please don’t add the “Az.” prefix when specifying the module name. The cmdlets in Az.Tool.Installer add this prefix automatically.

1.2 Install All Az Modules in both General and Preview Release

• Install-AzModule -Repository PSGallery –AllowPrerelease [-Name Storage,Network]

This cmdlet also provides an easy way to install preview modules with the parameter “-AllowPrerelease”. When this parameter is present, the cmdlet installs all the Az modules with the latest versions whether they are in general release or preview without parameter “-Name”. It tries to retrieve all the Az modules in the target repository before the installation and consequently takes more time to run.

The “-Name” parameter allows you to limit the installation of preview modules to only the ones specified.

Note: The preview version of Az.Accounts is not going to be installed because it contains breaking changes which break other modules.

2. Uninstall-AzModule

Table-2 The Syntax of Uninstall-AzModule

2.1 Uninstall the Modules Attached to a Specific Az

• Uninstall-AzModule -Repository PSGallery [-RequiredVersion 4.6.1] [-Name Storage,Network]

As an enhancement of Uninstall-Module, Uninstall-AzModule helps to uninstall the numerous modules related to Az. Firstly, you can uninstall the modules attached to a release version of Az using “-RequiredVersion” or “-MinimumVersion” or “-MaximumVersion”. If any of these are specified, by default the cmdlet removes all the modules attached to the Az version that fills the condition. If “-Name” is specified, the cmdlet removes the modules attached to the Az version for the specified name or list of names. If no version condition is provided explicitly, the latest Az version in the repository provided (PSGallery in the example above) is used.

2.2 Uninstall all the Az Modules Installed

• Uninstall-AzModule -Repository PSGallery –AllVersion [-Name Storage,Network]

To remove all the Az modules on your local machine, you can specify “-AllVersion”. You can provide a list of modules with “-Name” to remove all the versions of the specified modules.

2.3 Uninstall Preview Modules

• Uninstall-AzModule -AllowPrerelease [-Name Storage,MySql] [-AllVersion]

The cmdlet also enables you to remove preview modules in three different ways. By default, when “-AllowPrerelease” is specified, it removes all the modules’ latest version whether they’re in general release or preview. By using “-Name”, you can limit the uninstallation to the specified modules. Further, -AllVersion removes all versions of the modules related to Az.

Note the difference between “Uninstall-AzModule -AllowPrerelease –AllVersion” and “Uninstall-AzModule –AllVersion”. The later only removes the modules in general release. If you want to clean up your environment thoroughly, you can run “Uninstall-AzModule -RemoveAzureRm -AllowPrerelease –AllVersion”.

3. Update-AzModule

Table-3 The Syntax of Update-AzModule

3.1 Generate a Report on What to Update

• Update-AzModule -WhatIf [-Name Accounts,Storage, Network]

The advance of “Update-AzModule” compared to “Update-Module” is to provide you with a table of what modules to update and how to update them. “-WhatIf” prevents the cmdlet from performing the operation on the modules. It can be used to generate a report of what would happen without performing the operation. With “-Name”, you can check only the modules you are interested in. This also makes it complete faster. Table 1 shows an example of the “To-Update” report.

Table-4 The Report of Modules Can be Updated

3.2 Perform a replacement Update

• Update-AzModule [-Name Compute,Storage,Network] [-RemovePrevious]

After viewing the report, you would like to perform the update. If you only want to update some of the modules, you can provide a list of module names with “-Name”. If you want to update all the modules that have newer versions, you can run “Update-AzModule”. If you want to replace the currently installed version with the latest version, specify “-RemovePrevious”.

Note that “Update-AzModule” only updates the modules currently installed. If you provide the name of a module not on your local machine, the cmdlet does nothing. Note: This cmdlet cannot be used to update preview modules. If, for example, Az.Storage releases a preview version newer than the existing general release version, run “Install-AzModule –Name Storage -Repository PSGallery –AllowPrerelease” to install the latest preview.

IV. Limitation and Future Work

PowerShell Core can import modules from the Windows PowerShell path, however, it has no rights to write to that path. This causes a conflict when you are trying to install a module with a version existing in the Windows PowerShell path. Therefore, Az.Tool.Installer warns you of the installation of Azure PowerShell in the Windows PowerShell path when you use it in PowerShell Core. One solution is to grant PowerShell Core administrator rights and another way is only to use Azure PowerShell either in Windows PowerShell or PowerShell Core. Further, PowerShell Core is recommended.

Another obvious issue of Az.Tool.Installer is the performance. As it accesses the remote repository frequently to fetch the information and install the modules, you experience a delay for several minutes while the cmdlet runs. We are going to optimize it in a future release.

Your feedback on Az.Tool.Installer is important to us. If you have any questions or suggestions, feel free to reach out to us on GitHub.

Functional testing in the Update Staging Lab

by Scott Muniz | Sep 21, 2020 | Uncategorized

This article is contributed. See the original author and article here.

Software vendors: You can now perform custom functional tests—using the test framework of your choice – via the self-serve Update Staging Lab portal. When we initially launched the service, we offered out-of-box tests, a pre-defined set of tests driven through standardized scripting. This, however, could not achieve full test coverage for many ISVs. Hence, in response to your feedback, we are providing a welcome addition to the experience the ability for you to upload automated functional tests.

To use this feature, the basic steps are:

1. Upload your files (binaries, dependencies and scripts) as a single .zip package.
2. Choose if you want to reboot the test virtual machines (VMs) at various points of execution.
3. Manage available options for your scripts.
4. Choose when to apply the Windows update on the VM during execution.

I’ll now go into more detail about each step of the process.

Upload a functional test package

To get started, navigate to the Upload page, select Upload new application under Application catalog on the left-side navigation menu of the Update Staging Lab portal in Azure. From there:

Tab 1 – Enter basic information. Provide the name and version of your application. In the Type of test option, select “Functional tests”. Note that the Out-of-Box (OOB) option is required by default.

Tab 2 – Upload the components of your package. Upload a zip file with your entire test (binaries, dependencies, scripts etc). See aka.ms/usl-package-outline for details. (Note: Both the Out-of-Box test scripts and the Functional test contents should be placed into the same zip file). Currently, the file size is limited to 2GB.

Tab 3 – Configure the Out-of-Box and Functional test tasks. Here, choose the path(s) to the PowerShell scripts that will install, launch, close, and uninstall your application (for Out-of-Box) as well as the path(s) to all your custom scripts to perform your functional test. (Note: A script to uninstall your application is optional). Currently, you can upload between 1 and 8 scripts for your functional tests. (Comment on this post if you need more scripts!)

(Optional) Configure a restart after installation. Some applications require a restart after installation. Select Reboot After Execution for the specific Script in the Tasks tab if you’d like a restart to be conducted after the execution of that script you.

Tab 4 – Choose when the Windows update gets installed: The application of the Windows Update patch is done before any script of your choice. It is recommended that you install a Windows update after app installation to closely mimic your real-world application use scenarios.

Tab 5 – Review and create the package. Once you have completed the steps listed above, select Create to finish the uploading process.

See the verification status of your package

Once your package has been created, we run an initial test to install, launch, close, and uninstall your application. This allows us to verify that your package can install on our service error-free.

The verification process could take up to 24 hours. Once verification is complete, you can see the status in the Manage packages menu, which would be one of two entries:

1. Verification succeeds. The package will be automatically tested against pre-release Windows updates for the OS builds you selected.
   or
2. Verification fails. You will need to investigate the reasons for the failure, fix the issue, and re-upload your package.

You will also be notified of either outcome via the notification icon in the Azure portal.

Thanks to our existing Update Staging Lab users for their patience as we built this functionality into the service. We will continue to enhance and evolve the service, so please feel free to reach out to uslpreview@microsoft.com with feedback! We’ll also keep you updated on more functionality as it becomes available in the coming months.

Feature update validation now available on Update Staging Lab

by Scott Muniz | Sep 21, 2020 | Uncategorized

This article is contributed. See the original author and article here.

Need insight on how your applications will perform with the next release of Windows 10—prior to it being available in market? Don’t want to maintain an environment to validate new Windows features? Want to run your validation tests against Windows Insider Program builds in our Azure environment?

Feature update validation on Update Staging Lab can help you achieve all these and more!

Check out the step-by-step outline below to find out how to access this new capability in the Update Staging Lab service. (Not yet taking advantage of Update Staging Lab? Sign up here to join the preview.)

To get started with feature update validation in Update Staging Lab, upload your applications (and related files) through the self-service onboarding portal. As you fill out the Test details:

1. Select Feature Update as your OS update type:

   

2. Choose the Windows Insider Channel against which you want your application validated. (We currently support the Insider Beta Channel.)

   

3. Select an in-market release of Windows 10 as the baseline for your test (and resulting insights!) and provide the other details required to onboard your package successfully.

   

4. To view the results and insights from the validation of your application against pre-release Windows 10 feature updates, visit Feature Updates Test Results.

   

We look forward to hearing your comments/feedback on this new feature as we enhance and evolve our service to meet your validation needs. You can also email us at USLPreview@microsoft.com.