What’s new: The new Azure Sentinel Notebooks experience is now in public preview!

by Scott Muniz | Sep 22, 2020 | Azure, Technology, Uncategorized

This article is contributed. See the original author and article here.

We are happy to announce the public preview for the new and revamped customizable Jupyter notebook experience running on the Azure Machine Learning (AML) platform for analyzing your security data, all within a secure Azure cloud environment!

The new user experience provides an updated interactive UI with Intellisense for improved productivity, support for existing Jupyter and JupyterLab experiences, dedicated notebook compute, as well as point-in-time notebook snapshots and a notebook file explorer for easy notebook collaboration. In addition, take advantage of built-in security analytics via Jupyter notebook templates and the MSTICPy Python library help jumpstart your security analytics and operations.

Whether you are a seasoned security analyst with extensive Python and Jupyter experience, or just starting out, you can immediately start experiencing these benefits by adding Jupyter notebooks to your threat defender arsenal. 

We highly recommend you check out the Getting started with Azure Sentinel Notebooks video and the official documentation to get started.

New intuitive and approachable UI

A new UI experience based on the open source Nteract project. This simple and intuitive UI focuses on delivering simplicity and ease-of-use with full IntelliSense and inline error highlighting directly in your notebooks, drag-and-droppable cells, individual tabs for each notebook, inline toolbars and less clutter. Support for Jupyter and JupyterLab experiences and 10X faster Azure Sentinel notebook launch times.

Improved collaboration and versioning

Easily share notebooks and other artifacts with other security analysts across your team and/or organization.  A new notebook file explorer to browse your notebooks and your team’s notebooks in one place making it easier to collaborate.  Revert changes or review prior data by using the new check-point feature to take point-in-time notebook snapshots.

Managed and flexible compute with additional security features

Pay only for the resources you consume with fully managed dedicated cloud-based compute for executing your notebook workloads.  Terminal access to your notebook compute. Ability to install custom Jupyter kernels (such as PowerShell and C#). Azure Resource Manager (ARM) templates for compute deployments (article). Additional security features such as RBAC and SSH policy options available today with VNET support coming in the fall.    

Happy threat hunting and investigation!

Azure Security Benchmark v2 is now available with expanded security control assessments

by Scott Muniz | Sep 22, 2020 | Azure, Technology, Uncategorized

This article is contributed. See the original author and article here.

Today we are pleased to announce the Azure Security Benchmark v2. To accelerate the security of your cloud adoption journey, Microsoft has developed the Azure Security Benchmark (ASB). The benchmark is designed to provide clarity on security best practices and controls for configuring and operating Azure and Azure Services.

ASB v2 builds on the work of ASBv1 and includes these updates:

• Mapping of NIST SP 800-53 controls (in addition to existing CIS control framework v7.1)
• Mapping of security stakeholders to benchmark recommendations
• Expansion and restructure of controls to make them clear and actionable
• Integration of Azure Security Compass guidance

Vision for azure security guidance

We have learned that securing Azure means different things to different roles in the organization and have built a system of integrated security guidance. Each of these are aligned together to simplify your security journey:

• Microsoft Security Best Practices – Recommendations for securing all assets in your enterprise, typically integrated into security architectures and strategies.
• Azure Well-Architected Framework – Guidance for workload owners to architect workloads that meet goals for security, performance, cost, and more.
• Cloud Adoption Framework (CAF) – Guidance for cloud adoption initiatives to plan and execute on a strategy that meets organizational goals for security, cost management, reliability, and more (includes Enterprise Scale Landing Zone reference implementation and automation)
• Azure Security Benchmark – Prescriptive best practices and controls to guide all roles in the security organizations on securing Azure and Azure Services

Over the course of the last few months, the teams working on ASB, Enterprise Scale Landing Zone in CAF, Azure Security Compass, Azure Security Top 10 Best Practices, and Microsoft Best Practices have been working to consolidate and align all this guidance together to make it simpler and easier for you to rapidly secure your Azure resources.

As with all our guidance, we would love to hear your feedback on how this is working for you and how we can improve it. You can reach us by sending us email.

What’s new in ASB v2?

In addition to mapping and aligning all the guidance together, the team focused on these improvements in ASBv2:

• Mapped to NIST SP 800-53 Controls: NIST SP 800-53 is one of the most used control frameworks in the Industry, so we updated the ASB controls to map with the NIST controls. Now you can use ASB to meet the NIST requirements in Azure and can monitor the requirements in Azure Security Center Compliance dashboard. The benchmark and mapping to NIST controls is also available in excel format for easy consumption.
• Identified Security Stakeholders: We added security stakeholders to each recommendation to help you bring in the right people in your organization to plan, approve, or implement it. The stakeholders are identified by their roles and responsibility definitions from the CAF.
• Updated and restructured the security controls to provide more clarity: We made changes to the Azure security controls to make them actionable and more effective. A few examples are:
  • Updated the controls to provide clear guidance on security outcomes rather than just technical configurations.
  • Created a Governance and Strategy control to guide your strategic planning and governance strategy for security.
  • Updated the Logging and Threat detection control to focus more on the outcome of threat detection rather than just collection of logs.
  • Added Privileged Access as a separate control to provide clear recommendations on the critically important discipline of protecting privileged accounts (which can have an outsized business impact if compromised).
  • Added the Posture and Vulnerability Management control to provide clear guidance on how to monitor and improve your cloud security posture.
  • Identified a number of opportunities to modernize legacy security approaches with modern cloud approaches in Network Security, Endpoint Security, Logging and Threat detection, and more.
  • Updated and added controls to ensure a full lifecycle view for all controls including Data Protection, Asset Management, Identity Management, and Backup and Recovery.

What’s coming next?

Here is a brief overview of upcoming features:

• Monitoring of ASB v2 recommendations: Today, you can use the Azure Security Center Regulatory Compliance Dashboard to monitor your live Azure environment status with all the Azure Security Benchmark controls. In upcoming weeks, Security Center will be fully integrated with ASB v2, automatically monitoring your environment with policies implementing the complete ASB v2 control set by default. 
• Implementing the Benchmark recommendations: In coming weeks, we will be publishing the ASB v2 blueprint which will help you to implement and enforce the benchmark requirements. Today you can achieve that by using Enterprise scale landing zone to get a head start and use it to implement the Benchmark recommendations.
• Security Baselines based on ASB v2: So far, we have published security baselines for 50 Azure services based on ASB v1. These baselines provide service guidance on how you can meet the Benchmark requirements for a specific service. Azure customers today use these baselines as part of their cloud service assessment process. In upcoming months, we will be updating these baselines and adding more service baselines based on the ASB v2 recommendations.
• Control framework targeted next: After CIS v7.1 and NIST SP 800-53 control mapping, we are working on adding mapping of PCI DSS control requirements in coming months. This will help you to meet PCI DSS control requirements using the Azure Security Benchmark.

Call to Action

You can get started now with planning and implementing the Azure Security Benchmark v2, automate deployment with  Landing Zone, and monitor status using the Azure Security Center Regulatory Compliance Dashboard.

We want to thank the multiple teams within Microsoft, contributors from the Azure community, and NIST for the help with ASBv2 effort!

If you would like to participate in improving the benchmark or provide feedback, please send us an email. We would love to hear your success stories and feedback on how to make it better!

Azure Arc enabled Kubernetes with GitOps

by Scott Muniz | Sep 22, 2020 | Azure, Technology, Uncategorized

This article is contributed. See the original author and article here.

These days, it seems like Kubernetes is one of the most popular conversation topics in the world of Cloud and modern applications. The question is not if your organization uses Kubernetes, it is when the organization will use it.

This post will focus on the need for standardization and not if Kubernetes is meeting your business and technical requirements or not. So now we established the assumption that you already use it, let’s talk about a couple of challenges.

Challenges #1 – Sprawling

If you are in the process of modernizing your applications and adopting cloud-native design patterns, you know, like “next-gen” scalability, availability, security, etc. you probably have some notion of why Kubernetes. But the challenge, in this case, is the Kubernetes cluster sprawl that is about to hit you.

Rather you are building it yourself on-premises, using the gazillion Kubernetes flavors out there, installing on bare-metal or deploying one of the managed Kubernetes offerings the cloud providers has to offer, the problem remains, and all of a sudden you are in the business of managing Kubernetes clusters, well, all over the place. I like to call this “my fleet is out of control” :)

Challenge #2 – “I am drifting away…”

You got your clusters, good for you! But how do you keep all these clusters configured the way you left it?! Don’t you want them to be all meet your configuration baseline?! You do!

It’s not just your clusters that matter because after all, the applications are what drives the business!

Looking at both situations, you can see a recurring theme. This is really about making sure no configuration, rather on the cluster and/or the applications deployed on it are drifting away. You want this because otherwise, you might be facing an outdated application or a cluster that is not meeting for example your security needs.

Azure Arc enabled Kubernetes + GitOps == Wining

Now that we addressed these couple of challenges, we can talk a solution – enters Azure Arc enabled Kubernetes with GitOps configurations.

By extending or “stretching” the Azure Resource Manager (ARM) control plane, we are able to project your Kubernetes clusters which are deployed OUTSIDE of Azure as 1st class citizens inside Azure next to existing resources, for example, as you can see in the figure below, Azure Kubernetes Service (AKS) clusters reside next to Azure Arc ones.

By doing so, you get a single interface to rule them all which is the start of the solution to challenge #1 – the sprawl of clusters.

Azure Arc enabled Kubernetes clusters alongside AKS clusters

Projecting the clusters is the fundamental building block and now you apply GitOps Configurations for these clusters. Azure Arc with Kubernetes and GitOps is not scary as one might think, the concept and the flow are very straight forward.

Generally speaking, GitOps with Kubernetes is about deploying your applications based on Git repository which represents the “source of truth” or the baseline for this app deployment.

It relays on a Kubernetes Operator, which Is the Flux Operator in the Azure Arc Kubernetes case to “listen” if changes are being made on the baseline, meaning the repo. If such changes occur, the operator will initiate a rolling update Kubernetes deployment, deploying the new Pods and terminating the old one.

This can be done against a standard Kubernetes YAML manifest or a Helm charts release, using also the Help Operator with conjunction to the Flux one (which also gets deployed automatically).

Application deployment GitOps flow with Azure Arc enabled Kubernetes

1. Existing Kubernetes clusters are already deployed
2. Azure Arc Kubernetes connected cluster is created
3. The user creates Azure Arc Kubernetes cluster GitOps Configuration
4. The Flux Operator (and optionally the Helm Operator) is deployed on the cluster and starts” listening” to the git repository with the user’s application code
5. The Flux operator initiates the user’s application deployment on the cluster, representing the current desired state
6. User is updating the application (creating a new app version) and merge changes to the repository
7. Flux pick up a change to the Git repository
8. Flux operator initiates a new user’s application version deployment on the cluster while removing old version application pods, resulting in a new Desired State

Cluster-level Configuration vs. Namespace-level Configuration

Cluster-level Configuration

With Cluster-level GitOps Configuration, the goal is to have a baseline for the “horizontal components” or “management components” deployed on your Kubernetes cluster which will then be used by your applications. Good examples are Ingress Controller, Service Meshes, Security products, Monitoring solutions, etc. Having such deployments as part of your GitOps Configuration will assure your cluster is meeting the cluster baseline standards.

Namespace-level Configuration

With Namespace-level GitOps Configuration, the goal is to have the Kubernetes resources deployed only in the namespace selected. The most obvious use-case here is simply your application and it’s respective pods, services, ingress routes, etc. Having such deployments as part of your GitOps Configuration will assure your applications are meeting the Kubernetes applications baseline standards.

Azure Arc enabled Kubernetes GitOps Configurations

So, as you can see, by having the same GitOps Configurations for all your Kubernetes clusters, managed by Azure Arc you are solving challenge #2 and able to govern a potential cluster and application config and versioning drifts.

Get Started Today!

In this post we briefly touched on the power of using Azure Arc enabled Kubernetes alongside its native GitOps Configurations capabilities. Having all your Kubernetes clusters projected as Azure resources and have the same GitOps Configurations for all of them will allow you to gain much better control for both fleet management and deployment baselines as well as drift avoidance.

To get started, visit the Azure Arc Jumpstart GitHub repository, where you can find more than 30 Azure Arc deployment guides and automation, including how to deploy end-to-end GitOps flows against your Azure Arc enabled Kubernetes clusters as well as visit the official Azure Arc documentation page.

In addition, check out these additional great Azure Arc blog posts!

Azure Arc & Lighthouse: Managing IT Infrastructure Anywhere at-scale

by Scott Muniz | Sep 22, 2020 | Azure, Technology, Uncategorized

This article is contributed. See the original author and article here.

This blog post is co-authored by @Nikhil_Jethava and @LauraNicolas  

Modern organizations often manage diverse and complex IT infrastructures that frequently sprawl to multi-cloud environments.

Many enterprises have chosen pattern or vendor specific tools causing a ruptured management experience and an inconsistent approach to their operations. This problem is heightened with the pressure to innovate and deliver applications faster to the market, as well with the explosion of cloud native technologies and practices. The absence of single view and consistent tooling complicates the management for customer and partners alike.

Azure is focused on delivering innovation anywhere with a wide offering of hybrid services that meet customers and partners requirements as their environments become more complex. When Azure Lighthouse was introduced, it was another major step to address these challenges as it uncovered new possibilities for cross tenant management in the Azure platform with greater scale, visibility, and accuracy, turning the Azure Portal into a single control plane. With the addition of Azure Arc these cloud operations and practices can be extended to every workload and infrastructure, regardless of what it’s running or where is running.

Build a single view to manage across tenants.

Azure Lighthouse enables cross and multi-tenant management bringing greater scale and visibility into operations. The secret sauce behind Azure Lighthouse is the Azure Delegated Resource Management capability that logically projects resources from one tenant onto another and unlocks cross-tenant management with granular role based access and eliminates the need to do context switching.

Although Azure Lighthouse will work on any multitenant scenario, like customers that may have multiple Azure AD tenants (e.g. multiple subsidiaries or geographies in separate tenants)  and it is very valuable for partners, specially Managed Service Providers (MSPs) as they can realize efficiencies using Azure’s operations and management tools for multiple customers.

To illustrate this scenario, let’s take a look at Contoso who is responsible for the IT operations of three separate entities: Microsoft, Fabrikam and Fourth Coffee, each of them running Azure workloads on dedicated tenants. Azure Lighthouse enables Contoso to centrally manage resource inventories, access and identity, governance, monitoring and security across all the other three tenants. By aggregating all this data in a single view, Contoso can achieve consistency, security, and compliance for all the tenants while achieving greater operational efficiencies and building new offerings.

Governance and Compliance Management

With Azure Policy, Contoso can create, edit and apply policy definitions within the delegated subscriptions, they can also get a compliance snapshot that ensures that managed resources are compliant with corporate or regulatory standards from all three tenants having a full picture of the compliance status. Also, if Contoso develops new policies their intellectual property will be protected by using Azure Lighthouse as they can be centrally managed from their own central tenant.

The Azure Policy portal has been enhanced so you can select multiple scopes that will include a list of managed tenants and subscriptions:

Inventory Tracking and Management

Contoso has now the ability to develop multitenant queries using Azure Resource Graph to filter resources, leverage tags or track changes. The tenant ID can be returned in the query results, so the subscription and delegated tenant can be identified.

Monitoring and Alerting

Contoso can also get monitoring and security alerts across all of the tenant’s subscriptions, run multitenant queries using KQL and set up dashboards that provides valuable insights on the managed environments. There is no need to store logs from different entities into a shared log analytics workspace, Microsoft, Fabrikam and Fourth Coffee can  keep their logs on a dedicated workspace in their subscription, while Contoso gets delegated access to them and get insights from all tenants. Once again Contoso can choose the scope they want to work with in the portal.

Security and Compliance

Contoso can offer managed security services by centrally protecting Azure resources with Azure Security Center and Azure Sentinel they can provide proactive/reactive security best practices. Azure Security Center has cross-tenant visibility to manage security posture centrally and take actions on recommendations, detect threats, and harden resources.

Azure Sentinel when working with Lighthouse, can track incidents and attacks across tenants as well as define cross-tenant KQL queries.

Process Automation and Configuration Management

Azure Automation can be set up at scale, including runbook automation, desired state configuration and update management. Contoso can automate processes running custom scripts on the managed tenants while having their IP protected.

Resource Deployment at Scale

Lighthouse allows Contoso to not only operate but also deploy and configure Azure services on the managed tenants’ subscriptions. Taking care of their networking, storage, virtual machines, container environments and PaaS services. The management of operational tasks of those resources like backup or disaster recovery are very often handed off to specialists like Contoso that can centrally manage backup, restore and replication as well.

Extend Azure management across your environments.

Very often, enterprises have resources on-premises or on other clouds and they need to extend operations to those hybrid and distributed states.  Having built processes and offerings using Azure Services and Lighthouse it would be very powerful if those could be stretched to run across on-premises, other clouds, or the edge.

With Azure Arc your on-premises and other clouds deployments become an Azure Resource Manager entity and as such, servers, Kubernetes clusters or data services  can be treated as first-class citizens of Azure.  As any other ARM resource, they can be organized into resource groups and subscriptions, use tags, policies, assign RBAC and you can even leverage Azure Arc to onboard other services such as Azure Monitoring, Azure Security Center, Azure Sentinel or Azure Automation. 

Let’s revisit the Contoso scenario; Microsoft, Fabrikam and Fourth Coffee all have workloads on their on-premises datacenters or in other clouds. With Azure Arc Contoso can not only understand and organize the breadth of operations, but also extend and grow services and offerings provided in Azure into every corner of the portfolio. Using Azure hybrid management services with Azure Arc allows Contoso to adopt cloud-native practices everywhere and Lighthouse will provide the multitenancy required to have a single view into operations.

Governance and Compliance

Azure Policies can now be assigned to Azure Arc enabled servers and Kubernetes to entirely manage governance and guarantee corporate compliance. An initiative like the one shown here ‘Enable Azure Monitor for VMs’ will group not only Azure VMs but also Azure Arc enabled servers both Linux and Windows machines having a full compliance snapshot.

Inventory Management

Multitenant queries with Azure Resource Graph, can now also include Azure Arc enabled resources with filtering, using tags or tracking changes.

Hybrid Services Onboarding at Scale

Contoso can automate the deployment of agents and onboard Arc enabled resources into Azure Monitoring, Azure Automation, Azure Security Center or Azure Sentinel either by using Azure Polices or Azure Arc’s extension management capabilities. The extension management feature for Azure Arc enabled servers provide the same post-deployment configuration and automation tasks that you have for Azure VMs.

Contoso can also leverage policies to guarantee that all resources are properly onboarded into services like Azure Monitor by setting up remediation tasks that use the extension management feature, it will fix automatically any non-compliant resources.

Access Management

Auditability provided by Lighthouse is kept as Azure Arc supports RBAC and the Azure activity log will keep track of actions.

Application and Data Management at Scale

Contoso can use configuration as code and uniformly govern and deploy containerized application using GitOps-based configurations across on-premises, multi-cloud, and edge. Contoso can link a cluster to a Git repo that becomes the single source of truth for container deployments and applications, Azure Arc enabled Kubernetes will make sure there is no drift between Git and what is running in the cluster.

Azure Arc enabled data services allows Contoso to run Azure data services like Azure SQL Managed Instance and Azure Database for PostreSQL Hyperscale on any Kubernetes cluster with unified management and familiar tools.

With Azure Arc and Azure Lighthouse, Contoso is empowered to create cloud native management operations with no location boundaries.

Get Started

On this blog post we touched on a set of scenarios that are possible by combining Azure Arc with Azure Lighthouse and that will empower you to build reliable and at scale operations for hybrid and multi cloud environments with cross-tenant capabilities. To get started with Azure Lighthouse check out these links:

• To start Onboarding Azure Lighthouse
• To explore other cross-tenant capabilities check the Azure Lighthouse Samples repository
• For a full feature list of supported services and scenarios in Azure Lighthouse documentation.
• Check out Azure Chief Technology Officer Mark Russinovich’s blog for a deep under-the-hood view.

To get started with Azure Arc visit these links: 

• To explore automation, deployment guides and over 40 different Azure Arc scenarios, visit the Azure Arc Jumpstart. 
• A closer look at Azure Arc enabled servers: https://aka.ms/DKirbyIgnite2020BlogArc
  
• Azure Arc enabled Kubernetes Monitoring: https://aka.ms/ArcK8sMonitor 
• The Azure Arc Jumpstart Project: https://aka.ms/BlogArcJumpstart
• Azure Arc enabled Kubernetes with GitOps: https://aka.ms/ArcK8sGitOps