Notebook – This request is not authorized to perform this operation. , 403

This article is contributed. See the original author and article here.

This a quick post about this failure and how to fix: Error: org.apache.spark.sql.AnalysisException: java.lang.RuntimeException:
The operation failed: ‘This request is not authorized to perform this operation.’, 403

First, let’s just add some context : 

When you are working on synapse workspace with the managed identity you would need to give Storage Blob Data contributor permission to the workspace that represents the managed identity permission:

https://docs.microsoft.com/en-us/azure/synapse-analytics/security/how-to-grant-workspace-managed-identity-permissions

More information here: https://docs.microsoft.com/en-us/azure/synapse-analytics/security/synapse-workspace-managed-identity

Speaking of managed identities –quick review on it: “A common challenge when building cloud applications is how to manage the credentials in your code for authenticating to cloud services. You can use the identity to authenticate to any service that supports Azure AD authentication, including Key Vault, without any credentials in your code.”

More here: https://docs.microsoft.com/en-us/azure/active-directory/managed-identities-azure-resources/overview?toc=/azure/synapse-analytics/toc.json&bc=/azure/synapse-analytics/breadcrumb/toc.json

So there you are with your workspace with the managed identity permissions granted running a notebook to create a database on Spark.

Note you are the one running the notebook on Synapse Studio and you are not using any credentials as you are under Synapse workspace:

%%spark
spark.sql("CREATE DATABASE IF NOT EXISTS nyctaxi")

Brought to you by Dr. Ware, Microsoft Office 365 Silver Partner, Charleston SC.