Introducing Rich Reporting and Troubleshooting for Microsoft Playwright Testing

This article is contributed. See the original author and article here.

Today, we’re excited to introduce rich reporting and easy troubleshooting for the Microsoft Playwright Testing service!

Microsoft Playwright Testing is a managed service built for running Playwright tests easily at scale. Playwright is a fast-growing, open-source framework that enables reliable end-to-end testing and automation for modern web apps. You can read more about the service here.

Now, with this new Reporting feature users can publish test results and related artifacts and view them in the service portal for faster and easier troubleshooting.

 

Quickly Identify Failed and Flaky Tests

 

In the fast-paced world of web development, applications evolve rapidly, constantly reshaping user experiences. To keep up, testing needs to be just as swift. Playwright automates end-to-end tests and delivers essential reports for troubleshooting. The Reporting feature provides a streamlined dashboard that highlights failed and flaky tests, enabling you to identify and address issues quickly. This focused view helps maintain application quality while supporting rapid iteration.

 

Screenshot of test results filtered by failed and flaky tests

 

 

Troubleshoot Tests Easily using rich artifacts

 

As test suites grow and the frequency of test execution increases, managing generated artifacts becomes challenging. These artifacts are crucial for debugging failed tests and demonstrating quality signals for feature deployment, but they are often scattered across various sources.

The Reporting feature consolidates results and artifacts, such as screenshots, videos, and traces, into a unified web dashboard, simplifying the troubleshooting process. The Trace Viewer, a tool offered by Playwright, that helps you explore traces and allows you to navigate through each action of your test and visually observe what occurred during each step. It is hosted in the service portal with the test for which it is collected, eliminating the need to store and locate it separately for troubleshooting.

 

Screenshot of trace viewer hosted in the service portal

 

Seamless Integration with CI Pipelines

 

Continuous testing is essential for maintaining application quality, but collecting and maintaining execution reports and artifacts can be challenging. Microsoft Playwright Testing service can be easily configured to collect results and artifacts in CI pipelines. It also captures details about the CI agent running the tests and presents them in the service portal with the test run. This integration facilitates a smooth transition from the test results to the code repository where tests are written. Users can also access the history of test runs in the portal and gain valuable insights, leading to faster troubleshooting and reduced developer workload.

 

Screenshot of test result with CI information

 

Join the Private Preview

 

For current Playwright users, adding the Reporting feature with your existing setup is easy. It integrates with the Playwright test suite, requiring no changes to the existing test code. All you need to do is install a package that extends the Playwright open-source package, add it to your configuration, and you’re ready to go. This feature operates independently of the service’s cloud-hosted browsers, so you can use it without utilizing service-managed browsers.

We invite teams interested in enhancing their end-to-end testing to join the private preview of the Reporting feature. This feature is available at no additional charge during the private preview period. However, usage of the cloud-hosted browsers feature will be billed according to Azure pricing.

Your feedback is invaluable for refining and enhancing this feature. By joining the private preview, you gain early access and direct communication with the product team, allowing you to share your experiences and help shape the future of the product.

 

Interested in trying out the reporting feature and giving us feedback? Sign up here.

 

Check out Microsoft Playwright Testing service here. If you are new to Microsoft Playwright Testing service, learn more about it.

Brought to you by Dr. Ware, Microsoft Office 365 Silver Partner, Charleston SC.