by Contributed | Jul 2, 2021 | Technology
This article is contributed. See the original author and article here.
Welcome back to another blog post about the Azure Percept DK!
Previously
In the previous post – Azure Percept – First Look – you’ll remember that we had a first look at the percept and what it was all about. Well, in this post we’ll take a look at the Azure Percept Audio Module, which allows for the recognition of Custom Keywords and Commands (among other things).
What is the Percept Audio?
The Azure Percept Audio (sometimes called the Percept Ear) is a “System on a Module” or SoM, which is designed as the Audio Interface for Audio Processing at the edge for the Azure Percept.
Along with the Carrier Board, Azure Percept Studio, Microsoft LUIS and Speech, the system can recognise keywords and commands to control devices using voice at the edge. This works both online and offline with the aid of the Carrier Board.
Azure Percept Audio Specifications
The basic specs for the Azure Percept Audio SoM are;
- Four-microphone linear array and audio processing via XMOS Codec
- 2x buttons
- 3x LEDs
- Micro USB
- 3.5 mm audio jack
You can find the full specifications here
Who’s it for?
Like the Vision SoM, Microsoft clearly have a set of industries in mind for the Azure Percept Audio SoM;
- Hospitality
- Healthcare
- Smart Buildings
- Automotive
- Retail
- Manufacturing
With applications such as;
- In-room Virtual Concierge
- Vehicle Voice Assistant and Command/Control
- Point of Sale Services and Quality Control
- Warehouse Task Tracking
This becomes clear later when we look at the sample applications we can spin up in a minute.
Azure Percept Audio – Required Services
The Azure Percept Audio SoM makes use of a couple of Azure Services to process Audio;
LUIS (Language Understanding Intelligent Service):
LUIS is an Azure service which allows interaction with applications and devices using natural language.
Using a visual interface, we’re able to train AI models without the need for deep Machine Learning experience of any kind.
The Azure Percept uses LUIS to configure Custom Commands, allowing for a contextualised response to a given command.
Cognitive Speech:
Cognitive Speech is an Azure Service offering Text-to-speech, speech-to-text, speech translation and speaker recognition.
Supporting over 92 languages, this service can convert speech to text allowing for interactivity with apps and devices.
On the flip side, with support for over 215 different voices in 60 languages, the Speech Service can also convert Text to-Speech improving accessibility and interaction with devices and applications.
Finally, the Speech Service can also translate between 30 different languages, allowing for real-time translation using a variety of programming languages, which I think is a really cool use case.
The Percept uses this service amongst other things, to configure a wake word for the device, by default this is the word “computer“. (See Star Trek IV – The Voyage Home!).
Azure Percept Audio – Sample Applications
If we navigate to Azure Percept Studio, from the Overview Page we can select the “Demos & tutorials” tab at the top;
If we scroll to the bottom of this page, we have some links to some Speech tutorials and demos.
The first thing we’ll choose is “Try out voice assistant templates”. Clicking this link presents us with a fly out with a selection of templates to choose from;
You can see here a selection of Sample Templates that speak to where Microsoft expect the Percept Audio to be used. All of these a limited to voice commands interacting with an environment. I’ll speak later about some scenarios I’d actually like to be thought about that are outside of this use case.
For now, we’ll deploy one of these pre-backed samples and see how it works!
Azure Percept Audio – Hospitality Sample Template Setup
Choosing the “Hospitality” option, agreeing to the terms and continuing on, we’re shown the resource creation flyout.
Here we can select the subscription and resource group we’d like to deploy the various resources to.
We’re also prompted for an Application Prefix. This allows the template to create resources with unique ids.
We can then choose a region close to us. At the time of writing we can choose between West US and West Europe, but I imagine this will grow once the Percept starts getting towards GA. I was actually surprised at the choice of regions here with no East US, North Europe, and no APAC region at all.
Moving on, the last item we need to select is the “LUIS prediction pricing tier, which we can leave at “Standard”, as the free tier doesn’t support speech requests sadly.
Hitting the “Create” button, then begins the process of deploying the speech theme resources.
We’re then prompted that this process can take between 2 and 4 minutes to complete…. This only took a matter of seconds for me nicely…
Azure Percept Audio – Hospitality Sample Template Demo
Once the template has completed deploying we’re then shown a demo Hospitality environment.
We should also now have 3 blue LEDs showing on the Percept;
I found these LEDs to be super bright, such that I couldn’t stare directly at them without then being able to see three (or is it 5? Ha) dots like Picard in that episode of Next Gen. They light my whole office up at night practically!
The Percept Audio LEDs will indicate different statuses depending upon their colour and flash pattern;
LED
|
LED State
|
Ear SoM Status
|
L02 |
1x white, static on |
Power on |
L02 |
1x white, 0.5 Hz flashing |
Authentication in progress |
L01 & L02 & L03 |
3x blue, static on |
Waiting for keyword |
L01 & L02 & L03 |
LED array flashing, 20fps |
Listening or speaking |
L01 & L02 & L03 |
LED array racing, 20fps |
Thinking |
L01 & L02 & L03 |
3x red, static on |
Mute |
The LEDs are labelled as shown in the following picture, with L01 on the left of the SoM, L02 in the middle and L03 on the far right;
Returning to the Hospitality demo environment. The screen is split up into several sections.
At the top of the demo environment we have an toolbar containing;
- Create Custom Keyword
- Create Custom Command
- Get Started
- Learn More
- Feedback
- Troubleshoot
Just below that we have the current keyword and command and links to change them should we wish…. We’ll actually be looking at all of that in another blog post, so keep your eyes peeled!
On the left we have an interaction area where we can enter commands for the Percept to action.
On the right we have a visual representation of the current environment, which reflects the actions our commands invoke.
Audio Output
Before we try executing any commands, the Percept uses the Speech Service to convert it’s command responses to spoken word.
For us to be able to hear that, we’ll need to connect some speakers to the device.
The Percept has a 3.5mm audio jack output for exactly that purpose… Hooking up some relatively low powered portable speakers to the line out jack will allow us to hear the responses to our commands.
Executing Commands
We can now try executing some commands. The Custom Keyword or Wake Word for the Percept defaults to “Computer” (Where’s Scotty when you need him!), we can say that followed by one of a few commands which are applicable to this particular sample;
- Turn on/off the lights
- Turn on/off the TV.
- Turn on/off the AC.
- Open/close the blinds.
- Set temperature to X degrees. (X is the desired temperature, e.g. 75.)
I noticed, and perhaps due to my English accent, that it took a while for the Percept to recognise my pronunciation of “Computer”… I did try pronouncing it with an American Accent (and also asking it for the chemical formula for Plexiglass), but that didn’t seem to help.
Eventually it did work, and I quickly learnt how to say the word for a relatively repeatable wake up. I did notice that often, it would take quite a while to “Wake Up” the first time I issued the wake up word, and after that it would work quite quickly. I also noticed that, if I’d left it idle overnight that, in perfect IT Crowd style, I’d actually have to turn it off and back on to get it working again. When I get some time, I’ll raise some feedback with the team.
Once I’d mastered the wake word, all the other instructions worked pretty well…. You can see one of my failed attempts at the top here;
By instructing the Percept to turn on the TV, the simulation on the right would show the TV on, and so on through the commands.
The only command that didn’t work as intended was the “Set Temperature” command, which didn’t accept the actual temperature as a parameter to the command.
It turns out that this was because I was trying to set the temperature too low, you can only set it within a few degrees of the set temperature, otherwise it just doesn’t work.
My Plans
The first thing I’m going to try with this is hook it all up to a Raspberry Pi and recreate the Hospitality experience in real life. I think this would make a really cool demo for future talks… (Also, Microsoft, if you need somebody to come and create you a demo room in Seattle, just shout eh! ;) Haha).
I did ask the team about perhaps using the Percept Audio to detect things other than speech. As I mentioned in my previous post, I have a client in the Ecology and Wildlife ecosystem, and I’d love to perhaps train the Percept to recognise the sounds of wildlife maybe?
My Thoughts
Having spent time making Alexa Skills, Speech Studio is quite limited in comparison to the tools around Alexa, but it’s got everything we need at the moment to make reasonable speech based interaction apps.
I did find it frustrating that it would either not understand me, or be really slow to wake to the wake word… This makes demoing the unit a bit hit and miss and elicits the usual sniggers from attendees…. Ha.
Those points aside, I found the experience worked well, with the Sample Applications being a great example of some of the ideas Microsoft have in mind for this side of the Percept.
by Contributed | Jul 2, 2021 | Technology
This article is contributed. See the original author and article here.
Bird Detector Based on Microsoft Azure and Raspberry Pi
Jiong Shi has been a Microsoft MVP for twelve years, who is interested in Windows 10 IoT Core, Windows Embedded, Azure IoT and UWP. He is an associate professor in School of Computer and Electronic Engineering, Zhejiang Wanli University, China. He is an author of the book “Windows 10 IoT Application Developer Guide”, a blogger, and a speaker for Ignite China. Jiong is active in local developer community, speaking at technical conferences, writing articles on hackster, contributing to Windows IoT Core open-source projects, serving as a mentor and Microsoft Community Contributor. Follow him on Twitter @dearsj001
MICROSOFT ENDPOINT MANAGER: HOW TO CONFIGURE WINDOWS FIREWALL POLICIES
Silvio Di Benedetto is founder and CEO at Inside Technologies. He is a Digital Transformation helper, and Microsoft MVP for Cloud Datacenter Management. Silvio is a speaker and author, and collaborates side-by-side with some of the most important IT companies including Microsoft, Veeam, Parallels, and 5nine to provide technical sessions. Follow him on Twitter @s_net.
C#.NET: HOW TO UNZIP FOLDER
Asma Khalid is an Entrepreneur, ISV, Product Manager, Full Stack .Net Expert, Community Speaker, Contributor, and Aspiring YouTuber. Asma counts more than 7 years of hands-on experience in Leading, Developing & Managing IT-related projects and products as an IT industry professional. Asma is the first woman from Pakistan to receive the MVP award three times, and the first to receive C-sharp corner online developer community MVP award four times. See her blog here.
Teams Real Simple with Pictures: Getting Hands on with Profile+
Chris Hoard is a Microsoft Certified Trainer Regional Lead (MCT RL), Educator (MCEd) and Teams MVP. With over 10 years of cloud computing experience, he is currently building an education practice for Vuzion (Tier 2 UK CSP). His focus areas are Microsoft Teams, Microsoft 365 and entry-level Azure. Follow Chris on Twitter at @Microsoft365Pro and check out his blog here.
How to create Webinars with Microsoft Teams
Vesku Nopanen is a Principal Consultant in Office 365 and Modern Work and passionate about Microsoft Teams. He helps and coaches customers to find benefits and value when adopting new tools, methods, ways or working and practices into daily work-life equation. He focuses especially on Microsoft Teams and how it can change organizations’ work. He lives in Turku, Finland. Follow him on Twitter: @Vesanopanen
by Contributed | Jul 2, 2021 | Technology
This article is contributed. See the original author and article here.
This post was co-authored by Yinhe Wei, Runnan Li, Sheng Zhao, Qinying Liao, Yan Xia, and Nalin Mujumdar
An important element of language learning is being able to accurately pronounce words. Speech service on Azure supports Pronunciation Assessment to empower language learners and educators more. At the //Build 2021 conference, Pronunciation Assessment is announced generally available in US English, while other languages are available in preview.
The Pronunciation Assessment capability evaluates speech pronunciation and gives speakers feedback on the accuracy and fluency of the speech, allowing users to benefit from various aspects.
Comprehensive evaluation near human experts
Pronunciation Assessment, a feature of Speech in Azure Cognitive Services, provides subjective and objective feedback to language learners in computer-assisted language learning. For language learners, practicing pronunciation and getting timely feedback are essential for improving language skills. The assessment is conventionally driven by experienced teachers, which normally takes a lot of time and big efforts, making high-quality assessment expensive to learners. Pronunciation Assessment, a novel AI driven speech capability, is able to make language assessment more engaging and accessible to learners of all backgrounds.
Pronunciation Assessment provides various assessment results in different granularities, from individual phonemes to the entire text input. At the phoneme level, Pronunciation Assessment provides accuracy scores of each phoneme, helping learners to better understand the pronunciation details of their speech. At the word-level, Pronunciation Assessment can automatically detect miscues and provide accuracy score simultaneously, which provides more detailed information on omission, repetition, insertions, and mispronunciation in the given speech. At the full-text level, Pronunciation Assessment offers additional Fluency and Completeness scores: Fluency indicates how closely the speech matches a native speaker’s use of silent breaks between words, and Completeness indicates how many words are pronounced in the speech to the reference text input. An overall score aggregated from Accuracy, Fluency and Completeness is then given to indicate the overall pronunciation quality of the given speech. With these features, learners can easily know the weakness of their speech, and improve with target goals.
With Pronunciation Assessment, language learners can practice, get instant feedback, and improve their pronunciation. Online learning solution providers or educators can use the capability to evaluate pronunciation of multiple speakers in real-time.
Pearson uses Pronunciation Assessment in Longman English Plus to empower both students and teachers to improve the productivity in language learning, with a personalized placement test feature and learning material recommendations for students at different levels. As the world’s leading learning company, Pearson enables tens of millions of learners every year to maximize their success. Key technologies from Microsoft used in Longman English Plus are Pronunciation Assessment, neural text-to-speech and natural language processing. Check below video for a demo of the Longman English learning app.
BYJU chooses Speech service on Azure to build the English Language App (ELA) to their target geographies where English is used as the secondary language and is considered an essential skill to acquire. The app blends the best of pedagogy using state-of-the-art speech technology to help children gain command over language with ease in a judgement-free learning environment. With a conversation-first interface, this app enables students to learn, and practice English while working on their language skills in a fun, engaging and effective manner. BYJU’S is using the Speech to Text and Pronunciation Assessment capabilities to ensure that children master English with ease – to practice speaking and receive feedback on pronunciation with phoneme, word and sentence-level pronunciation and fluency scores. BYJU’S ELA assesses pronunciation of students through speaking games, identifies areas of improvement, and provides personalized and adaptive lessons to help students improve in their weak areas.
Mispronunciation detection and diagnosis
Mispronunciation Detection and Diagnose (MDD) is the core technique employed in Pronunciation Assessment, scoring word-level pronunciation accuracy, which provides judgement on miscues and contributes to the overall assessment. To provide precise and consistent result, Pronunciation Assessment employs the latest powerful neural networks for modelling, exploiting information from lower senone granularity to higher word granularity with the use of hierarchical architecture. This design enables Pronunciation Assessment to fully exploit the detailed pronunciation information from small patterns, making mispronunciation detection more accurate and robust. With 100,000+ hours training data on different accents, regions and ages, Pronunciation Assessment can also handle different scenarios with various users, for example, from kids to adults, from none-native speakers to native speakers, and provide trustable and consistent assessment performance.
Teams Reading Progress uses Pronunciation Assessment to help students improve reading fluency, after the pandemic negatively affected students’ reading ability. It can be used inside and outside of the classroom to save teachers’ time and improve learning outcomes for students. Learn how to get started.
“Reading Progress is built on the solid scientific foundation of oral repeated reading and close monitoring by the educator. It allows educators to provide personal attention to each student while at the same time dealing with a whole classroom full of students.”
— Tim Rasinski, Professor of Literacy Education at Kent State University
Cutting-edge free-style speech assessment
Pronunciation Assessment also supports spontaneous speech scenarios. Spontaneous speech, also known as free-style talk, is the scenario where speakers are giving speech without any prefixed reference, like in presentation and spoken language examination. Empowered with Azure Speech-to-Text, Pronunciation Assessment can automatically transcribe a given speech accurately, and provide assessment result on aforementioned granularities.
Pronunciation Assessment is used in PowerPoint coach to advise presenters on the correct pronunciation of spoken words throughout their rehearsal. When Presenter Coach perceives that you may have mispronounced a word, it will display the word(s) and provide an experience that helps you practice pronouncing the word correctly. You’ll be able to listen to a recorded pronunciation guide of the word as many times as you’d like.
Get started
To learn more and get started, you can first try out Pronunciation Assessment to evaluate a user’s fluency and pronunciation with the no-code tool provided in Speech Studio, which allows you to explore the Speech service with intuitive user interface. You need an Azure account and a Speech service resource before you can use Speech Studio. If you don’t have an account and subscription, try the Speech service for free.
Here are more resources to help you add speech to your educational applications:
Tags:
by Contributed | Jul 2, 2021 | Technology
This article is contributed. See the original author and article here.
SharePoint Framework Special Interest Group (SIG) bi-weekly community call recording from July 1st is now available from the Microsoft 365 Community YouTube channel at http://aka.ms/m365pnp-videos. You can use SharePoint Framework for building solutions for Microsoft Teams and for SharePoint Online.
Call summary:
Summer break and community call schedule updates reviewed. Register now for June trainings on Sharing-is-caring. PnP Recognition Program announced. Join the Viva Connections private preview! Update on SharePoint Framework v1.13.0 features – extensibility options with Viva Connections, Teams improvements, tooling updates, Store modernization and more. Released PnPjs for Client-side Libraries v2.6.0, CLI for Microsoft 365 v3.11.0 Beta, and PnP Modern Search v4.2.3 & v3.20.0. There were four PnP SPFx web part samples delivered in last 2 weeks. Great work!
Latest project updates include: (Bold indicates update from previous report 2 weeks ago)
PnP Project |
Current version |
Release/Status |
SharePoint Framework (SPFx) |
v1.12.1 |
v1.13.0 Preview in summer |
PnPjs Client-Side Libraries |
v2.6.0 |
v2.7.0 scheduled for July 9, v3.0.0 developments underway |
CLI for Microsoft 365 |
v3.11.0 Beta, v4.0.0 Preview |
v3.10.0 preview released |
Reusable SPFx React Controls |
v2.8.0 (SPFx v1.11), v3.2.1 (SPFx v1.12.1) |
|
Reusable SPFx React Property Controls |
v2.7.0 (SPFx v1.11), v3.2.0 (SPFx v1.12.1) |
|
PnP SPFx Generator |
v1.16.0 |
v1.17.0 on the way |
PnP Modern Search |
v4.2.3 & v3.20.0 |
July versions in progress |
The host of this call is Patrick Rodgers (Microsoft) | @mediocrebowler. Q&A takes place in chat throughout the call.
It appears fun is being had by all! Great to see everyone. Thanks for making this a great community.
Actions:
Demos:
Bringing your custom apps to Viva Connect mobile experience – three graduating levels or views of information and customizability in Viva Connections. Use SPFx to build an Adaptive Card Extension (ACE) that supports 2 views – Card View (level 1 basic information) and Quick View (level 2 more information) and a deep link to a Teams tab (level 3 full information). Future, seamlessly drill down from high level to detail – mobile and desktop experiences.
Copy PnP Search results web part settings – scenario: Many of your pages are based a template page you now need to update. After updating your template page, use this list view command set extension accessed from Pages Library, to copy the settings of the PnP search results web part present on a page to the settings of the PnP search results web part present on other selected pages.
Follow Documents in Modern Workspace with SPFx – many capabilities in 2 related samples – 1 extension, 1 web part. Both accomplish same objective: Easily identify/follow/unfollow a user’s key documents within Tenant and easily open them in Modern Pages (use extension) or Modern Pages and Microsoft Teams (use web part + extension). This solution uses the Out of Box Social functionality “Follow document“ and Microsoft Graph queries. Exhaustive feature list by sample.
SPFx web part samples: (https://aka.ms/spfx-webparts)
Thank you for your great work. Samples are often showcased in Demos.
Agenda items:
Demos:
Demo: Bringing your custom apps to Viva Connect mobile experience – Vardhaman Deshpande (Valo Intranet) | @vrdmn – 19:14
Demo: Copy PnP Search results web part settings – Anoop Tatti (Content and Code) | @anooptells – 31:24
Demo: Follow Documents in Modern Workspace with SPFx – André Lage (Datalynx AG) | @aaclage – 44:26
Resources:
Additional resources around the covered topics and links from the slides.
General Resources:
Other mentioned topics:
Upcoming calls | Recurrent invites:
PnP SharePoint Framework Special Interest Group bi-weekly calls are targeted at anyone who is interested in the JavaScript-based development towards Microsoft Teams, SharePoint Online, and also on-premises. SIG calls are used for the following objectives.
- SharePoint Framework engineering update from Microsoft
- Talk about PnP JavaScript Core libraries
- Office 365 CLI Updates
- SPFx reusable controls
- PnP SPFx Yeoman generator
- Share code samples and best practices
- Possible engineering asks for the field – input, feedback, and suggestions
- Cover any open questions on the client-side development
- Demonstrate SharePoint Framework in practice in Microsoft Teams or SharePoint context
- You can download a recurrent invite from https://aka.ms/spdev-spfx-call. Welcome and join the discussion!
“Sharing is caring”
Microsoft 365 PnP team, Microsoft – 1st of July 2021
by Contributed | Jul 1, 2021 | Technology
This article is contributed. See the original author and article here.
For Oracle 19c AWR reports for Multi-tenant DBs on Exadata reports, the %Busy, CPU, Cores, memory and other data isn’t present. These are vital data points many of us use to determine vCPU calculations in sizing.
Never fear- we still have the data provided in the AWR report we received from the customer to fill in these sections or at least calculate the value.
Memory, CPU, Cores
Most of the missing data is in the Exadata section of the report:
How Much CPU is Available?
Now, to create gather a % Busy for CPU usage from what we have in the AWR, we’ll need the following values:
Using this info, we can create the following calculation to discover the amount of CPU available on the RAC node in the Exadata:
CPU Count * Elapsed Time * 60 seconds = CPU_VAL
So, for this example, this was an AWR on a 32 CPU machine for 300 minutes.
32 * 300 * 60 seconds = 576000 seconds of CPU available.
Suddenly, the DB CPUs value doesn’t look so huge anymore, does it?
How Busy is the CPU for Oracle?
A disclaimer needs to be added as there are some holes in the AWR capture for CPU busy, especially around “wait for CPU”, but to come up with a % busy CPU, we have the numbers we need to create this value now that’s missing in the AWR.
(CPU_VAL – DB CPUs)/(DB CPUs) = % Busy CPU
Now we have our % Busy, which for this node of the RAC database on the Exadata was 24.65, or 25%.
If you’re lucky enough to have access to the customer’s database server, you could also just use this valuable post from my peep, Kyle Hailey and run a query to collect the info. Kyle also goes into the challenges and holes in the AWR data that if you want to dig in deep vs. the estimates we require for sizing, would require ASH data to fill in.
col metric_name for a25
col metric_unit for a25
SELECT metric_name, value,_metric_unit
FROM v$sysmetric
WHERE metric_name like ‘%Host CPU%’;
If you run into this challenge with an Oracle 19c sizing estimate, hopefully this post will help keep you from pulling your hair out!
Related: How to size an Oracle Workload in Azure.
Recent Comments