Speeding up recovery and VACUUM in Postgres 14

Speeding up recovery and VACUUM in Postgres 14

by | Mar 25, 2021 | Technology

This article is contributed. See the original author and article here.

One of the performance projects I’ve focused on in PostgreSQL 14 is speeding up PostgreSQL recovery and vacuum. In the PostgreSQL team at Microsoft, I spend most of my time working with other members of the community on the PostgreSQL open source project. And in Postgres 14 (due to release in Q3 of 2021), I committed a change to optimize the compactify_tuples function, to reduce CPU utilization in the PostgreSQL recovery process. This performance optimization in PostgreSQL 14 made our crash recovery test case about 2.4x faster.


 


The compactify_tuples function is used internally in PostgreSQL:


 



  • when PostgreSQL starts up after a non-clean shutdown—called crash recovery

  • by the recovery process that is used by physical standby servers to replay changes (as described in the write-ahead log) as they arrive from the primary server

  • by VACUUM


So the good news is that the improvements to compactify_tuples will: improve crash recovery performance; reduce the load on the standby server, allowing it to replay the write-ahead log from the primary server more quickly; and improve VACUUM performance.


 


In this post, let’s walk through the change to compactify_tuples, how the function used to work, and why the newly-rewritten version in Postgres 14 is faster.


 


Speedometer-image-for-Postgres14-vacuum-recovery-blog-1920x1080.jpg


 


Profiling the recovery process highlighted a performance problem


 


In PostgreSQL, the write-ahead log (called the WAL) contains sets of records with instructions and data to go with the instructions. These WAL records describe changes which are to be made to the underlying data. WAL is used to make sure each change to the data is durable in the event that PostgreSQL stops running before the underlying data is written to disk. When PostgreSQL restarts after a shutdown, all of the WAL since the last checkpoint must be replayed as part of the recovery process. The WAL is replayed in order, to reapply changes to pages which may not have been written out to disk before the database shutdown.


 


We did some profiling of the PostgreSQL recovery process after purposefully crashing the database after running a benchmark of an UPDATE-heavy, OLTP type workload. These profiles showed that much of the CPU load was coming from the replay of “HEAP2 CLEAN” WAL records. HEAP2 CLEAN records defragment pages in tables to remove free space left by dead tuples. HEAP2 CLEAN records are added to WAL whenever the page becomes full and more space is needed on the page.


 


A tuple is PostgreSQL’s internal representation of a row in a table. A single row may have many tuples representing it, but only one of these tuples will be applicable at any single point in time. Older running transactions may require access to older versions of the row (another tuple) so they can access the row as it was at the point in time when the transaction started. UPDATEs to rows cause new versions of the tuple to be created. Creating multiple versions of each row in this way is called multi-version concurrency control (MVCC.)


 


Removing unused space from heap pages


 


To understand what HEAP2 CLEAN does in PostgreSQL, we’ll need to peer into the heap page format. Let’s look at a typical (simplified) heap page with some tuple fragmentation:


 


 


Figure 1: A PostgreSQL heap page with fragmentation due to removed tuplesFigure 1: A PostgreSQL heap page with fragmentation due to removed tuples


 


We can see that after the page header comes an array of “items.” The items act as pointers to the start of each tuple. PostgreSQL writes tuples to the page starting at the end and works backwards. The page is full when the item array and tuple space would overlap.


 


You should also notice that the tuples at the end of the page are not exactly in the reverse order of the item pointers. Tuples 2 and 3 appear out of order here. Tuples can become out of order after some records have been updated in the page and an old item pointer gets reused.


 


We can also see that that there’s quite a bit of unused space on the page in figure 1. The unused space is due to VACUUM removing tuples. The HEAP2 CLEAN operation will get rid of this unused space.


 


In PostgreSQL 13 and earlier, the HEAP2 CLEAN operation would turn the above into:


 


Figure 2: A defragmented heap page in PostgreSQL before the performance improvement in PostgreSQL 14 that speeds up the recovery process and VACUUM.Figure 2: A defragmented heap page in PostgreSQL before the performance improvement in PostgreSQL 14 that speeds up the recovery process and VACUUM.


 


We can see that the empty space is gone and the tuples are now pushed up to the end of the page. Notice the tuples remain in the same order as they were, with tuples 2 and 3 remaining in the same swapped order.


 


How heap page compactification used to work, prior to PostgreSQL 14


 


The compactify_tuples function takes care of the page compactification operation for us. All the changes made are in the compactify_tuples function. Before this change, the compactify_tuples function would perform a sort on a copy of the items array. This sort allowed the tuples to get moved starting with tuples at the end of the page. In this case, the tuples are moved starting with tuple1, followed by tuple3, tuple2 and tuple4. To determine which tuples to move first, compactify_tuples used the generic C qsort function with a custom comparator function. Sorting the item array in reverse tuple offset order allowed the tuples to be moved starting at the tuples at the end of the page.


 


Since heap pages can contain up to a few hundred tuples, and due to how often pages can be compactified in UPDATE-heavy workloads, the qsort call could use a lot of CPU. The sort overhead was not as bad with pages just containing a few tuples.


 


Do we really need to sort? … Well, yes and no. If we moved tuples in item array order and didn’t sort, then we could overwrite tuples later in the page. For example, in figure 2, if we moved tuple2 toward the end of the page before moving tuple3, then we’d overwrite tuple3. When moving the tuples in-place like this, we must make sure we move tuples at the end of the page first. So, we must sort when doing this in-place move.


 


How to make HEAP2 CLEAN faster?


 


To speed up the page compactification done during HEAP2 CLEAN, we could have written a custom qsort function which inlines the comparator function. Creating a custom qsort would have reduced some function call overhead, but, no matter what we did, qsort would still average an O(n log n) complexity. It would be nice to get rid of the sort completely.


 


The qsort is only required so that we don’t overwrite any yet-to-be-moved tuples. So, instead of sorting, we could instead copy those tuples to a temporary in-memory buffer, that way the order we move the tuples does not matter.


 


The new version of compactify_tuples in PostgreSQL 14 completely eliminates the need to use qsort. Removing the sort allows us to move tuples to the end of the page in item-array order. The temporary in-memory buffer removes the danger of tuples being overwritten before they’re moved and also means that the tuples are put back in the correct order with the first tuple at the end of the page.


 


The new heap page format, after compactification in PostgreSQL 14, looks like this:


 


Figure 3: A newly compactified heap page after the performance improvement in PostgreSQL 14.Figure 3: A newly compactified heap page after the performance improvement in PostgreSQL 14.


 


Notice that tuples 2 and 3 have now swapped places and the tuples are now in backwards item array order.


 


The new Postgres 14 code is further optimized by having a pre-check to see if the tuples are already in the correct reverse item pointer offset order. If the tuples are in the correct order, then there is no need to use the temporary buffer. We then move only tuples that come earlier (working backwards) in the page than the first empty space. The other tuples are already in the correct place. Also, now that we put the tuples back into reverse item pointer order again, we hit this pre-sorted case more often. On average, we’ll only move half the tuples on the page. New tuples which generate a new item pointer will also maintain this order for us.


 


Having the tuples in reverse item order may also help some CPU architectures prefetch memory more efficiently than if the tuples were in a random order in the page.


 


How much faster is the recovery process now in PostgreSQL 14?


 


My test case used a table with two INT columns and a “fillfactor” of 85 with 10 million rows. Accounting for the tuple header, this allows a maximum of 226 tuples to fit on each 8 kilobyte page.


 


To generate some WAL to replay, I used pgbench, a simple benchmarking program that comes with PostgreSQL, to do 12 million UPDATEs to random rows. Each of the 10 million rows will receive an average of 12 updates. I then did an unclean shutdown of PostgreSQL and started it back up again, forcing the database to perform crash recovery. Before the performance improvement, crash recovery took 148 seconds to replay the 2.2 GB of WAL.



The new version of the compactify_tuplescode took the same test 60.8 seconds. With this change and the given test case, PostgreSQL’s crash recovery became about ~2.4x faster.


 

















  Before the change to compactify tuples After the change to compactify tuples Performance impact of compactify tuples change
Time to replay a
2.2 GB WAL as part of crash recovery		 148 sec 60.8 sec ~2.4X faster

 


Previously, before the compactify_tuples change, pages with large numbers of tuples were the slowest to compactify. This was due to the qsort taking longer with large arrays. After changing compactify_tuples the performance is much more consistent with varying numbers of tuples on the page. The change does still result in a small speedup on pages even with very few tuples per page. However, the change helps the most when the number of tuples per page is large.


 


VACUUM performance also improved in PostgreSQL 14


 


VACUUM (and autovacuum) happens to make use of the same code after they remove dead tuples from a heap page. So, speeding up compactify_tuples also means a nice performance improvement for vacuum and autovacuum, too. We tried performing a VACUUM of the table updated my benchmark from earlier and saw that VACUUM now runs 25% faster in PostgreSQL 14 than it did before the compactify_tuples change. Previously it took 4.1 seconds to VACUUM the table and after the change that time went down to 2.9 seconds.


 


Speeding up the recovery process also means that physical standby servers are more likely to keep up pace and replay the primary’s WAL as quickly as it is being generated. So making this change to compactify_tuples also means that standby servers are less likely to lag behind the primary.


 


So the recovery process and vacuum will be faster in PostgreSQL 14—and there’s more work in progress, too


 


The change to compactify_tuples does help improve the performance of the recovery process in many cases. However, it’s also common for the recovery process to be bottlenecked on I/O rather than CPU. When recovering databases which are larger than the system’s available RAM, recovery must often wait for a page to be read from disk before any changes can be applied to it. Luckily, we’re also working on a way to make the recovery process pre-fetch pages into the kernel’s page cache so the physical I/O can take place concurrently in the background rather than having the recovery process wait for it.



For those of you who want an even bigger peek into how Postgres development happens, you can subscribe to any of the Postgres mailing lists. The PostgreSQL mailing list archives are also searchable. And if you want to learn even more about this compactify_tuples performance fix, here is the text of the actual commit to PostgreSQL 14, plus a link to the discussion on this optimization topic on the pgsql-hackers mailing list.


 

Author: David Rowley <drowley@postgresql.org>	 
Date: Wed, 16 Sep 2020 01:22:20 +0000 (13:22 +1200) 
commit 19c60ad69a91f346edf66996b2cf726f594d3d2b 

Optimize compactify_tuples function 

This function could often be seen in profiles of vacuum and could often 
be a significant bottleneck during recovery. The problem was that a qsort 
was performed in order to sort an array of item pointers in reverse offset 
order so that we could use that to safely move tuples up to the end of the 
page without overwriting the memory of yet-to-be-moved tuples. i.e. we 
used to compact the page starting at the back of the page and move towards 
the front. The qsort that this required could be expensive for pages with 
a large number of tuples. 

In this commit, we take another approach to tuple compactification. 

Now, instead of sorting the remaining item pointers array we first check 
if the array is presorted and only memmove() the tuples that need to be 
moved. This presorted check can be done very cheaply in the calling 
functions when the array is being populated. This presorted case is very 
fast. 

When the item pointer array is not presorted we must copy tuples that need 
to be moved into a temp buffer before copying them back into the page 
again. This differs from what we used to do here as we're now copying the 
tuples back into the page in reverse line pointer order. Previously we 
left the existing order alone.  Reordering the tuples results in an 
increased likelihood of hitting the pre-sorted case the next time around. 
Any newly added tuple which consumes a new line pointer will also maintain 
the correct sort order of tuples in the page which will also result in the 
presorted case being hit the next time.  Only consuming an unused line 
pointer can cause the order of tuples to go out again, but that will be 
corrected next time the function is called for the page. 

Benchmarks have shown that the non-presorted case is at least equally as 
fast as the original qsort method even when the page just has a few 
tuples. As the number of tuples becomes larger the new method maintains 
its performance whereas the original qsort method became much slower when 
the number of tuples on the page became large. 

Author: David Rowley 
Reviewed-by: Thomas Munro 
Tested-by: Jakub Wartak

 

MidDay Café Open QA Sessions

MidDay Café Open QA Sessions

by | Mar 25, 2021 | Technology

This article is contributed. See the original author and article here.

HLS Mid-Day Café3.pngFor the next four weeks MidDay Café is moving to an Open Q&A format. Have questions around Microsoft Teams voice, Viva, SharePoint, Teams administration, and more? Bring ‘em to MidDay Café and let’s get those questions answered. In addition to Open Q&A the MidDay Café crew will have the weekly news, upcoming events, plus some surprise tips and tricks!


Join us Monday’s 3/29, 4/5, 4/12, and 4/19 for MidDay Café Open QA Sessions.


MidDay Café Open QA Sessions Agenda:



  • Welcome and Introductions.

  • Mid-Day Café News and Events

  • Tips and Tricks of the Week.

  • Open Q&A

  • Wrap Up


For the Event:



Keep up to date with MidDay Café:



 Thanks for visiting – Michael Gannotti   LinkedIn | Twitter


Michael GannottiMichael Gannotti

Learn about Microsoft & SAP solutions for Financials, Governance and Cybersecurity

by | Mar 25, 2021 | Technology

This article is contributed. See the original author and article here.

Discover how Microsoft enabled a highly regulated SAP landscape for the new MS-Federal entity, how to adopt a Zero Trust Security architecture for SAP deployments, understand implementation of SAP S/4HANA Central Finance, unlocking business value with cloud economics, and more!


 


Microsoft & SAP jointly participated at the SAPInsider event on Finance, GRC and Cybersecurity from Mar 13-15. In case you missed the event, watch the sessions on-demand now through Apr 15. Click on the links below and use MICROSOFTGUEST to register.


 


Azure Marketplace new offers – Volume 125

Azure Marketplace new offers – Volume 125

by | Mar 25, 2021 | Technology

This article is contributed. See the original author and article here.











We continue to expand the Azure Marketplace ecosystem. For this volume, 102 new offers successfully met the onboarding criteria and went live. See details of the new offers below:



































































































































































































































































































































































































































Applications
adenin Digital Assistant.png

adenin Digital Assistant: Developed by adenin TECHNOLOGIES, Digital Assistant is an AI-powered chatbot that connects to your business applications to boost productivity, reduce workplace distractions, and provide notifications for upcoming tasks.
AlmaLinux 8.3 Beta.png

AlmaLinux 8.3 Beta: This preconfigured image from ProComputers.com provides a minimal installation of AlmaLinux 8.3 Beta. AlmaLinux is an open-source, community-driven project that is mainly used as a common base system on top of which other appliances are built and tested.
Anti-Microbial Stewardship-Zevac.png

Anti-Microbial Stewardship-Zevac: Zevac from Circle of Life Healthcare is an AI-powered antimicrobial stewardship solution designed to help hospitals stop the inappropriate or unnecessary use of antibiotics.
Apache Solr Helm Chart.png

Apache Solr Helm Chart: Bitnami offers this Helm Chart of Apache Solr for deployment on Kubernetes. Apache Solr is a powerful, open-source enterprise search platform built on Apache Lucene.
Ataccama ONE Data Quality Management.png

Ataccama ONE Data Quality Management: Ataccama ONE on Microsoft Azure automates data quality management at any scale. Get the tools and functionalities required for a wide range of use cases, including standardization and cleansing, de-duplication, reporting, and data masking.
Ataccama ONE Master Data Management.png

Ataccama ONE Master Data Management: Ataccama ONE combines features from prominent master data management (MDM) styles and gives you the freedom to manage your master data your way. Consolidate data from various sources, create a 360-degree view of any domain, publish changes directly to golden records, and more.
AuditeCloud.png

AuditeCloud: AuditeCloud extends Microsoft Teams messaging and collaboration capabilities to enable customer service operators to communicate with customers via any social network or social messenger. This application is available only in Italian.
Bella Education Suite.png

Bella Education Suite: Available in Portuguese and English, Bella Education Suite uses artificial intelligence to help improve the quality of your institution’s educational services. Gain actionable insights into teaching methods and differentials while exploring new ways to teach and evaluate.
BigBlueButton- Online Learning.png

BigBlueButton: Online Learning: BigBlueButton is an open-source web conferencing system for online learning that enables teachers to share audio, video, screens, and more in real time with their students. A simple API makes it easy to integrate BigBlueButton with existing apps and systems.
BitwardenRS- Password storing and management.png

BitwardenRS: Password storing and management: Linnovate Technologies’ BitwardenRS is an unofficial Bitwarden server implementation written in Rust. Featuring end-to-end encryption, BitwardenRS provides a safe and easy way for teams and individuals to store and share sensitive data from any device.
CHASE Platform.png

CHASE Waste management SaaS platform: Available only in Traditional Chinese, the Chase waste management platform combines artificial intelligence with big data analytics to help organizations deal with their waste quickly and efficiently.
climb2.png

Climb: Developed by RockStep Solutions, Climb is a SaaS collaboration suite for managing in vivo drug discovery data and operations. Empower lab managers, study directors, and scientists with on-demand data access and full situational awareness of their research.
Concept Encoder.png

Concept Encoder: Concept Encoder is an AI-powered solution that uses vectorization of words and documents to digitize and analyze natural sentences in healthcare and life sciences. This application is available only in Japanese.
Contract Management Software for Dynamics 365 CE.png

Contract Management Software for Dynamics 365 Customer Engagement: Contracts 365 for Microsoft Dynamics 365 is a fully integrated contract lifecycle management module that facilitates the synchronization of data between Contracts 365 Standard or Enterprise Edition and Dynamics 365. Reduce cycle times, facilitate transparency, and more.
Core Intranet.png

Core Intranet: Good internal communications tools are key in today’s workplace. Sigma’s Core Intranet is a fast, intuitive intranet and digital workspace solution that delivers customizable information to employees across an organization.
Crestron Flex.png

Crestron Flex: Built on the Crestron XiO Cloud, Crestron Flex is a portfolio of IoT-enabled automation and control devices that help organizations improve productivity, optimize space utilization, reduce energy use, and make informed decisions about current and future resource needs based on real-time data.
Crestron XiO Cloud.png

Crestron XiO Cloud: The Crestron XiO Cloud helps increase productivity, improve facility efficiencies, and strengthen decision-making by enabling organizations to provision, monitor, and manage IoT-enabled devices that control room scheduling, environment, and collaboration technologies.
Data Mobility and Migration.png

Data Mobility and Migration: Data Dynamics’ Unified Unstructured Data Management Platform leverages automation to help you move data from heterogenous storage environments to Microsoft Azure. Optimize your storage infrastructure by ensuring you have the right data in the right location at the right time.
Dealer Management System.png

Dealer Management System: Excellon 5 is a cloud-based dealer management solution for managing complex sales, distribution, and service networks. Leverage multi-channel workloads and manage all your leads, customers, call centers, and campaigns with Excellon 5.
Digital Adoption Solution for Office 365 and Azure.png

Digital Adoption Solution for Microsoft 365 and Azure: Vitalyst’s Digital Adoption Solution helps facilitate a deep learning culture and Microsoft product usage at scale across your organization. Features include end user skilling and one-on-one coaching to improve organizational mastery of Microsoft products and technologies.
Evventa.png

Evventa: Evventa on Microsoft Azure is an events, protocol, and ceremonials management suite of web-based solutions. It automates the ins and outs of complex events and protocols to facilitate work completion while ensuring a high level of security across branches and departments.
F5 Platform.png

F5 Platform: Created for capital-intensive midsize and large industrial enterprises, F5 Platform is a powerful data analytics framework that empowers you to build business applications quickly and provide valuable insights for your business. This application is available only in Russian.
factoHR HRMS Application.png

factoHR HRMS Application: factoHR is a mobile-first, hire-to-retire HCM platform that enables businesses to automate their daily HR functions. Simplify talent discussions and gain actionable insights to help build a great place to work and achieve business objectives faster.
Fiscal Attach.png

Fiscal Attach: Fiscal Attach is an optical character recognition (OCR) service that reads files sent by your service providers and turns them into JSON or XML files to be imported into your organization’s management system. This application is only available in Portuguese.
Eamified Esport Fan Monetization.png

Gamified Esport Fan Monetization: Omnicoach helps e-sport businesses boost fan monetization by connecting brands with fans. The AI-based analytics solution provides measurable data points in gaming videos and enables the creation of customized quests and rewards linked to moments in games to engage users.
Ghost- publishing platform based on Node.js.png

Ghost: Publishing Platform Based on Node.js: Ghost is a modern publishing solution featuring a rich editor designed to provide great writing experiences. Leverage image galleries, embeds, Markdown, HTML, and more while developing direct relationships with your audience and generating revenue.
Guide Modern.png

Guide Modern: Guide Modern from Sigma is a quality management system that helps employees and managers work systematically across their business. All information is stored in SharePoint Online and is searchable from Microsoft 365 and SharePoint.
Incident Management System.png

Incident Management System: CogniTensor’s Incident Management System is a combination of correlation analytics and predictions dashboards designed to enable emergency response teams to track and manage any incidents that may arise.
Insighta.png

Insighta: Available from eSense Software, Insighta extends your enterprise performance management solution to mobile devices along with analytics, actionable data, and reports. Gain quick, on-the-go insights into your organization’s projects, budgets, invoices, and more.
Inspector Cloud Camera 2.png

Inspector Cloud Camera 2: Powered by computer vision, Inspector Cloud is a SaaS solution that recognizes products to help CPG companies check their shelf KPIs. Gain visibility into your sales team’s efficiency and reduce manual inspection times and human error with Inspector Cloud.
Intelligent Forecasting and Scenario Modeling.png

Intelligent Forecasting and Scenario Modeling: Powered by Microsoft technologies, EY Global’s Intelligent Forecasting and Scenario Modeling solution is an AI forecasting engine that leverages machine learning to address business planning needs and challenges across an enterprise.
Intelligent Health Management Platform.png

Intelligent Health Management Platform: Saans Health Labs’ intelligent health management platform uses artificial intelligence, machine learning, and analytics to help healthcare providers predict the onset of chronic conditions and their associated costs.
Intranet Out-of the-Box Solution.png

Intranet Out-of the-Box Solution: Cambay’s Intranet Out-of-the-Box Solution is a unified collaboration platform for corporate communication and document management. Built on Microsoft Azure, the cloud-first, mobile-first solution offers seamless mobile experiences to keep users engaged from any device.
Invested.ch.png

invested.ch: Designed to level the financial playing field by enabling everyone to profit from growing economies, invested.ch delivers intuitive tools to support those who have never bought a stock. Learn how to start owning stocks simply, transparently, and affordably.
JiVS IMP.png

JiVS IMP: JiVS IMP helps you manage the lifecycle of company data independently of other systems and applications. Reduce operating and migration costs, along with the total cost of ownership, with support for more than 2,000 business objects, including SAP and non-SAP systems.
Kortext.png

Kortext: Universities use Kortext to deliver course texts to students around a single aggregated bookshelf. The platform empowers universities to deliver connected, enhanced learning experiences to help ensure no student is left behind.
KrakenD API Gateway.png

KrakenD API Gateway: KrakenD is an open-source API gateway that aggregates and manipulates multiple data sources to provide you with exactly the API your products need while offering a premium user experience and advanced performance.
LoRaWAN No-Code End to End Solution Builder.png

LoRaWAN No-Code End-to-End Solution Builder: Webee’s fully managed, no-code IoT solution enables Microsoft Azure customers to connect and manage wireless devices that use long-range, low-power wide-area network (LoRaWAN) connectivity. Accelerate IoT application development using Azure services.
LTI SWITCH.png

LTI SWITCH: Hosted on Microsoft Azure, SWITCH is an insurance book transfer and rollover solution for eliminating manual tasks during the book transfer process. The SaaS platform uses Azure Cognitive Services to build core business logic and is available on a subscription basis.
mAdvisor Automl.png

mAdvisor AutoML: Marlabs Global Development Center’s mAdvisor is an automated machine learning (AutoML) platform that translates data from enterprise systems into meaningful insights and predictions. Identify new revenue streams and enhance productivity and customer experience.
ManageEngine ADManager Plus (1 Domain 5 HelpDesk).png

ManageEngine ADManager Plus (1 Domain 5 Help Desk): ADManager Plus is an integrated Active Directory (AD), Exchange Server, Microsoft 365, and Skype for Business management and reporting solution that enables you to securely delegate organizational unit and group-based AD tasks to help your desk technicians.
ManageEngine ADManager Plus (1 Domain).png

ManageEngine ADManager Plus (1 Domain): ADManager Plus is an integrated Active Directory (AD), Exchange Server, Microsoft 365, and Skype for Business management and reporting solution that enables you to securely delegate organizational unit and group-based AD tasks to help your desk technicians.
ManageEngine PAM360 (10 Admins, 25 Keys).png

ManageEngine PAM360 (10 Admins, 25 Keys): ManageEngine PAM360 is an enterprise-grade unified privileged access management solution. It enables password administrators and privileged users to gain granular control over critical IT assets, such as passwords, SSH keys, digital certificates, and license keys.
Manty Decision.png

Manty Decision: Manty Decision is a data visualization platform for public administrations and local authority managers. Available only in French, Decision connects to a community’s data sources and centralizes all data in a unified data lake to drive informed decision-making.
Maxis I.Protect BaaS.png

Maxis I.Protect BaaS: Maxis I.Protect is a data protection lifecycle management engine that enables users to run self-service automation processes from a web console. Streamline and automate critical data management tasks, including backup and disaster recovery, to Microsoft Azure or on-premises.
mDERMS.png

mDERMS: Developed by mPrest Systems, mDERMS on Microsoft Azure is a utility orchestration and optimization platform. The solution helps organizations manage distributed energy resources while optimizing their dispatch and facilitating end-to-end visibility and monitoring.
Minecraft (Java) Server on Windows Server 2016.png

Minecraft (Java) Server on Windows Server 2016: Virtual Pulse offers this preconfigured image of a Minecraft Java game server on Windows Server 2016. Prepare for a grand adventure as you build, mine, battle mobs, and explore the ever-changing Minecraft landscape.
Minecraft (Java) Server on Windows Server 2019.png

Minecraft (Java) Server on Windows Server 2019: Virtual Pulse offers this preconfigured image of a Minecraft Java game server on Windows Server 2019. Prepare for a grand adventure as you build, mine, battle mobs, and explore the ever-changing Minecraft landscape.
moreReception.png

moreReception: Available only in Japanese, moreReception greatly reduces the burden and cost of reception for multi-tenant buildings. The Microsoft Teams-based solution calls corporate personnel via a simple touch panel to help ensure their customers aren’t left waiting in the lobby.
Morphisec Keep for Azure.png

Morphisec Keep for Azure: The Morphisec Keep server threat prevention platform for Microsoft Azure uses moving-target defense to proactively prevent cyberattacks on your physical and virtual servers without relying on signatures, attack patterns, or machine learning algorithms.
MySQL 5.7 with Red Hat 7.9.png

MySQL 5.7 with Red Hat 7.9: This preconfigured image from Cognosys provides MySQL 5.7 with Red Hat 7.9. MySQL is a popular open-source relational SQL database management system used for developing web-based applications.
MySQL 8.0 with Red Hat 7.9.png

MySQL 8.0 with Red Hat 7.9: This preconfigured image from Cognosys provides MySQL 8.0 with Red Hat 7.9. MySQL is a popular open-source relational SQL database management system used for developing web-based applications.
nC Microsoft Teams Navigator.png

nC Navigator for Microsoft Teams : novaCapta’s Navigator app for Microsoft Teams extends Teams by enabling the mapping of hierarchies and dependencies with structured, multi-level navigation. This application is available only in German.
NGINX with Red Hat 7.9.png

NGINX with Red Hat 7.9: Cognosys offers this preconfigured image of NGINX with Red Hat 7.9. NGINX is an all-in-one API gateway, cache, load balancer, web application firewall, and web server.
Novari MIRM.png

Novari MIRM: Novari’s Medical Imaging Requisition Management (MIRM) solution eliminates paper and dramatically improves requisition workflows for radiologists, technologists, clerical staff, and hospital leadership. Streamline your requisition workflows for all modalities with MIRM.
Office FlexiSpace-Office Workplace Reservations.png

Office FlexiSpace – Office Workplace Reservations: Office FlexiSpace on Microsoft Azure enables your employees to quickly and easily reserve a workplace in the office. Employees can book rooms via a web browser, a chatbot in Microsoft Teams, or a mobile app. This application is available in Russian.
OfficeExpert EPM.png

OfficeExpert EPM: OfficeExpert EPM provides detailed performance data on all Microsoft Teams calls and meetings. The integrated solution gathers information from localized agents running on computer endpoints to provide accurate metrics for measuring performance and identifying UX issues.
Payment Analysis System.png

Payment Analysis System: CogniTensor’s Payment Analysis System is a combination of correlation analytics and prediction dashboards designed to help you address various finance objectives. Features include AI-powered predictions, a real-time reminder system, and a comprehensive grading system.
Preppio.png

Preppio: Preppio for Microsoft Teams serves as a culture and process assistant for everything related to employee onboarding. It automates communication between new hires and HR managers to help ensure an engaging, high-quality onboarding process.
Purple's WiFi Analytics and Marketing Automation.png

Purple’s WiFi Analytics and Marketing Automation: Purple on Microsoft Azure is a cloud-based platform designed to turn Wi-Fi networks into revenue-generation tools for businesses through social media integration, intuitive analytics, and profile-based marketing.
Ramco HR Software and Global Payroll Suite.png

Ramco HR Software and Global Payroll Suite: Ramco Systems’ Global Payroll is a comprehensive HR solution that uses artificial intelligence and machine learning to detect payroll fraud, provide facial recognition-based time and attendance functionality, and more.
Recorded Future Express.png

Recorded Future Express: Recorded Future Express provides instant access to threat intelligence for web-based resources. The extension provides real-time risk scores for domains, IP addresses, hashes, and URLs, enabling you to quickly understand and act on potential threats.
Recruitment Smart.png

Recruitment Smart: Recruitment Smart offers a deep-tech HR platform that automates the end-to-end talent supply chain for organizations. The solution integrates seamlessly with resident applicant tracking systems to help you find, qualify, and hire talent according to your business needs.
Rendezvous Occupancy Sensors.png

Rendezvous Occupancy Sensors: Rendezvous Occupancy Sensors use real-time data to deliver an accurate breakdown of desk and space use in your office. See how your space is being used, set up real-time alerts to monitor when space is underused and overused, and more.
Repstance Advanced Edition.png

Repstance Advanced Edition: Repstance is an enterprise-class data replication and migration tool for Microsoft SQL Server, Oracle, and Snowflake that enables you to deploy, distribute, update, and use your most valuable asset – your data.
Safe Steps.png

Safe Steps: Safe Steps is an IoT solution that helps businesses manage employee and visitor safety during the COVID-19 pandemic by automating social distancing and contract tracing. Wearable devices use BLE-based proximity technology to detect social distancing violations and notify users.
SafeGUARD Backup and Archiving.png

SafeGUARD Backup and Archiving: SafeGUARD is a policy-driven, cloud-based data management solution that enables users to discover, manage, and protect all their data, including data stored in Microsoft 365 applications.
SAS SaaS Contract Analytics.png

SAS SaaS Contract Analytics: The process of legal reasoning and decision-making relies heavily on information stored in text. SAS SaaS Contract Analytics uses natural language processing to automate tasks such as due diligence, contract review, and legal discovery and help you save time and money.
Sayint Contact Center Intelligence.png

Sayint Contact Center Intelligence: Tech Mahindra’s Sayint on Microsoft Azure is a speech analytics platform that uses artificial intelligence, machine learning, and natural language processing to uncover actionable insights from customer conversations.
Sendmarc.png

Sendmarc: Sendmarc uses the DMARC email authentication protocol to protect your domain from email phishing and impersonation domains. Gain visibility into all email sent from your domain, drive compliance, and control how your domain is used.
SFTP - OpenSSH FTP Server on CentOS 7.8.png

SFTP – OpenSSH FTP Server on CentOS 7.8: This preconfigured image from Virtual Pulse provides SFTP – OpenSSH FTP server on CentOS 7.8. SFTP is a secure file transfer protocol that protects against password sniffing and man-in-the-middle attacks while supporting the security and authentication functionalities of SSH.
Sharecare Health Security Vaccination Platform.png

Sharecare Health Security Vaccination Platform: Built using Microsoft Cloud for Healthcare and Azure Health Bot, Sharecare is a vaccine distribution platform that addresses healthcare technology infrastructure needs for scheduling, onboarding, tracking, and monitoring COVID-19 vaccinations.
Sinequa for Azure.png

Sinequa for Azure: Optimized for Microsoft Azure, Sinequa’s intelligent search platform surfaces the right information to the right people at the right time regardless of source, format, or language. Accelerate innovation, foster collaboration, ensure compliance, and increase productivity.
SOP Management SaaS.png

SOP Management SaaS: DHC VISION SOP & Training Manager supports all aspects of regulatory-compliant document management, from creation and editing to approval, distribution, training, and archiving. This application is available in German and English.
TeamViewer.png

TeamViewer: TeamViewer is a comprehensive remote access, remote control, and remote support solution that works with most desktop and mobile platforms, including Windows, macOS, Android, and iOS. TeamViewer uses Azure Active Directory to manage user access, provision accounts, and enable SSO.
Ticketing.png

Ticketing: kitameraki’s ticketing app for Microsoft Teams helps boost company efficiency and productivity by handling your service desk incidents and requests. The industry-agnostic solution benefits any team that provides an internal service to the rest of the company.
TraceSCAN.png

TraceSCAN: TraceSCAN on Microsoft Azure is a comprehensive, customizable contact tracing solution that aims to halt the spread of COVID-19 by alerting users who have been in close contact with individuals who have tested positive for the disease.
Turicode - Document Processing & Data Extraction.png

Turicode – Document Processing & Data Extraction: Turicode’s intelligent document processing solution enables users to train machine learning models to automate the processing of all kinds of documents, including customs documents, salary statements, and pension fund statements.
UST Healthy Spaces.png

UST Healthy Spaces: UST Healthy Spaces provides enterprise-class tools to ensure proper health, safety, protective equipment, and sanitary conditions are maintained across your organization. The solution provides entry control, PPE detection, sanitation management, and more.
UST Omni.png

UST Omni: UST Omni provides end-to-end supply chain visibility at a granular SKU level during every stage of your organization’s supply chain. Be more demand-driven and market-driven, improve your service levels, and make your supply chain more responsive to market dynamics with UST Onmi.
UST SmartOps.png

UST SmartOps: UST SmartOps is an intelligent process automation platform that automatically learns and improves your business processes while digitizing and compressing repetitive human-intensive tasks.
Utilmate - Your Utility Billing Solution.png

Utilmate – Your Utility Billing Solution: Utilmate is an industry-compliant data-to-cash utility billing solution for suppliers of electricity, gas, water, and solar energy. Utilmate features a streamlined set-up process to get you up and running quickly.
VNS3 5.x Firewall Router VPN.png

VNS3 5.x Firewall/Router/VPN: Cohesive Networks’ VNS3 firewall/router/VPN solution on Microsoft Azure enables you to deliver improved security, connectivity, and compliance while minimizing the complexity of your organization’s IT environment.
VNS3 NATe - NAT Gateway Appliance.png

VNS3 NATe – NAT Gateway Appliance: Built on Cohesive Networks’ VNS3 Platform, VNS3 NATe is a preconfigured NAT gateway appliance. All VNS3 controllers can be live upgraded to provide the capabilities your cloud use case requires.
Wipster Enterprise.png

Wipster Enterprise: Wipster Enterprise is a comprehensive review and approval platform for video. Streamline your video review cycles with on-frame commenting, version control and comparison, media sharing, and support for a wide variety of files.
WordPress with Red Hat 7.9.png

WordPress with Red Hat 7.9: This preconfigured, ready-to-run image from Cognosys provides WordPress 5.6.1 with Red Hat 7.9. WordPress is a popular open-source content management system based on PHP and MySQL.
WordPress with Red Hat 8.png

WordPress with Red Hat 8: This preconfigured, ready-to-run image from Cognosys provides WordPress 5.6.1 with Red Hat 8.3. WordPress is a popular open-source content management system based on PHP and MySQL.
XpenseCo Travel and Expense.png

XpenseCo Travel and Expense: The XpenseCo app for Microsoft Dynamics 365 makes it easy to book travel by providing visibility into travel and expense spending and enabling finance teams to provide spend forecasts by employee, project, and business unit.
YourVoice.png

YourVoice – User feedback and idea management: YourVoice for Microsoft 365 drives continuous improvement across your organization by enabling you to manage employee feedback and suggestions for improving your service offerings. This application is available in German and English. 
Zero Trust Secure Remote Access.png

Zero Trust Secure Remote Access: Block Armour offers an enterprise-grade zero-trust solution to facilitate secure and compliant remote access for your hybrid or distributed IT environment. Empower teams of remote workers without sacrificing security or productivity.

Consulting services
App Identity Evaluation Workshop- 1 Day.png

App Identity Evaluation Workshop: 1 Day: This App Identity Evaluation workshop from KiZAN Technologies will help you simplify the management and security of your applications by establishing the use of Azure Active Directory as a single solution for your cloud and on-premises applications.
Azure Admin for AWS SysOps- 2-Day Workshop.png

Azure Admin for AWS SysOps: 2-Day Workshop: Dynamics Edge’s two-day offering is designed for AWS SysOps administrators who are interested in learning how Microsoft Azure is different from AWS and how Azure is administered. The workshop covers Azure networking, compute, storage, and governance.
Azure Cloud Security Assessment- 4 Weeks.png

Azure Cloud Security Assessment: 4 Weeks: Gain comprehensive insight into your cloud security posture along with a pathway to achieving security maturity with Sourced’s Microsoft Azure Cloud Security Assessment. Deliverables include a roadmap with recommendations to meet your cybersecurity risk-management objectives.
AzureReady Health Check- 2-Week Assessment.png

AzureReady Health Check: 2-Week Assessment: 3Cloud’s AzureReady Health Check is designed to evaluate the cost, security, scalability, and manageability of your organization’s Microsoft Azure environment. Make sure your Azure environment is configured properly, backed up, and secure.
Cloud4C Azure Managed Services - Security.png

Cloud4C Azure Managed Services – Security: Cloud4C Services combines technology, intelligence, and security operation center expertise to offer comprehensive managed security services for Microsoft Azure. Get real-time monitoring and event analysis for protection from advanced tactics, techniques, and procedures.
Data Platform Modernization- 5-Day Assessment.png

Data Platform Modernization: 5-Day Assessment: Data is your organization’s most valuable asset. Lucient’s four-step process (assessment, justification, project execution, and governance) helps you understand the different options for your data, including each option’s pros and cons and associated costs.
HCL PICASSO- Half-Day Workshop.png

HCL PICASSO: Half-Day Workshop: HCL Technologies’ half-day workshop will cover use-case scenarios that take advantage of the Microsoft Connected Vehicle Platform on Microsoft Azure and Azure IoT Hub. Learn how to maximize efficiency with real-time intelligence, increase speed to market, and more.
Managed Security.png

Managed Security: Armis offers a 24×7 managed cyber risk mitigation service that integrates Microsoft’s cloud-native security information and event management solution, Azure Sentinel. Proactively detect and respond to security incidents and defend your organization against attacks.
Migrate SQL Workloads to Azure- 2-Day Workshop.png

Migrate SQL Workloads to Azure: 2-Day Workshop: Learn about Azure SQL Database and what is required to migrate your MySQL and PostgreSQL workloads to Azure SQL Database in Dynamics Edge’s two-day workshop. Understand the benefits and limitations of each target platform and how each can be used to fulfill your requirements.
Office Line Lighthouse.png Office Line Lighthouse: Office Line will prepare your environment for a Microsoft Azure migration by implementing an Azure Landing Zone, then deliver proactive maintenance, update management, and recommendations for improvement.
PEI Azure Managed Services.png

PEI Azure Managed Services: Have you recently deployed solutions to Microsoft Azure and want help managing them? Performance Enhancements provides full production support and management for your environment, no matter where your organization is in its Azure journey.
Quest Technology Management Azure Lighthouse.png

Quest Technology Management Azure Lighthouse: Quest Media & Supplies offers a host of managed services for Quest clients using Microsoft Azure, including consulting, infrastructure and cost management, and security audits.