In this blog, we are going to cover Azure Synapse Link, Azure Synapse Link for Cosmos DB, Benefits, HTAP Scenarios, Security, and its limitations.
Topics we’ll cover:
- Azure Synapse Link
- Azure Synapse Link For Azure Cosmos DB
- Benefits
- HTAP Scenarios
- Security
- Limitations
- Enable Synapse Link For Azure Cosmos DB Using Portal
What Is Azure Synapse Link
Azure Synapse Link is a cloud-native hybrid transactional and analytical processing (HTAP) capability that enables near real-time analytics over operational data in Azure Cosmos DB. Azure Synapse Link creates a tight seamless integration between Azure Cosmos DB and Azure Synapse Analytics.
Azure Synapse Link for Azure Cosmos DB
Azure Synapse Link enables no Extract-Transform-Load (ETL) analytics in Azure Synapse Analytics against your operational data at scale. Business analysts, data engineers, and data scientists can now use Synapse Spark or Synapse SQL interchangeably to run near real-time business intelligence, analytics, and machine learning pipelines.
Source: Microsoft
Azure Synapse Analytics provides both a SQL Serverless query engine for querying the analytical store using familiar T-SQL and an Apache Spark query engine for leveraging the analytical store using your choice of Scala, Java, Python, or SQL and provides a user-friendly notebook experience.
Benefits Of Azure Synapse Link
Azure Synapse Link for Azure Cosmos DB offers several advantages such as:
- Reduced complexity with No ETL jobs to manage: You can run large-scale analytics against the analytical store, from Azure Synapse Analytics, without additional data transformation.
-
No impact on operational workloads: The analytical workload is independent of the transactional workload traffic, not consuming any of the provisioned throughputs of your operational data.
-
Cost-effective: Azure Cosmos DB analytical store follows a consumption-based pricing model, which is based on data storage and analytical read/writes operations and queries executed.
-
Analytics for locally available, globally distributed, multi-region writes: Azure Cosmos DB provides the state-of-the-art capability to run the globally distributed analytical workloads along with transactional workloads in an active-active manner.
HTAP Scenarios For Synapse Link
HTAP stands for Hybrid Transactional/Analytical Processing and is a database model that can ingest high volumes of data while also enabling analytical processing concurrently. Existing architectures usually enable one or the other. NoSQL is excellent at horizontally scaling and enabling ingests of high volumes of data but is severely limited in deep analytics.
You can add indices to obtain your data in different ways, but this duplicates data and takes up more resources, and increases cost. Even with these added indices, you will never keep ahead of the data needs of your business.
1.) Azure Cosmos DB analytical store
Azure Cosmos DB analytical store is a column-oriented representation of your operational data in Azure Cosmos DB. The analytical store automatically picks up high-frequency inserts, updates, deletes in your transactional workloads in near real-time, as a fully managed capability (“auto-sync”) of Azure Cosmos DB. No change feed or ETL is required.
2.) Integration with Azure Synapse Analytics
With Synapse Link, you can now straightforwardly interface with your Azure Cosmos DB compartments from Azure Synapse Analytics and access the analytical store with no different connectors.
This integration empowers the accompanying HTAP situations for various clients:
- A BI Engineer needs to demonstrate and distribute a Power BI report to get to the live functional information in Azure Cosmos DB straightforwardly through Synapse SQL.
- A Data Analyst, who needs to get experiences from the functional information in an Azure Cosmos DB holder by questioning it with Synapse SQL, read the information at scale, and consolidate those discoveries with different information sources
Security In Azure Synapse Link
Azure Synapse Link for Azure Cosmos DB is designed to help meet these security requirements through the following features:
- Network isolation using private endpoints: Network segregation is finished involving separate oversaw private endpoints for each store, inside oversaw virtual organizations in Azure Synapse work areas
- Data encryption with customer-managed keys: Azure Synapse Link just backings designing client oversaw keys utilizing your Azure Cosmos DB record’s overseen character. You should arrange your record’s overseen personality in your Azure Key Vault access strategy prior to empowering Azure Synapse Link.
- Secure key management: Rather than utilizing the Azure Cosmos DB account keys inline in Spark occupations or SQL scripts, Azure Synapse Link gives safer abilities.
When using Synapse serverless SQL pools, you can query the Azure Cosmos DB analytical store by pre-creating SQL credentials storing the account keys, and referencing these in the open row function.
Limitations Of Azure Synapse Link
- Azure Synapse Link for Azure Cosmos DB is upheld for SQL API and Azure Cosmos DB API for MongoDB. It isn’t upheld for Gremlin API, Cassandra API, and Table API.
- Getting to the Azure Cosmos DB examination store with Azure Synapse Dedicated SQL Pool right now isn’t upheld.
- Empowering Synapse Link on existing Cosmos DB compartments is just upheld for SQL API accounts. Synapse Link can be empowered on new compartments for both SQL API and MongoDB API accounts.
- Reinforcement and re-establish of your information in the analytical store aren’t upheld as of now.
Enable Synapse Link For Azure Cosmos DB Using Portal
1. Sign in to the Azure portal. And search for the Azure Cosmos DB.
2. Create a new Azure account, or select an existing Azure Cosmos DB account.
3. Navigate to your Azure Cosmos DB account and open the Features pane. Next, it prompts you to enable the synapse link on your account. Select Enable.
4. Your account is now enabled to use Synapse Link. Turning on Synapse Link does not turn on the analytical store automatically. Once you enable Synapse Link on the Cosmos DB account, enable the analytical store on containers to start using Synapse Link.
Related/References
- Microsoft Certified Azure Data Engineer Associate | DP 203 | Step By Step Activity Guides (Hands-On Labs)
- Exam DP-203: Data Engineering on Microsoft Azure
- Microsoft Azure Data Engineer Associate [DP-203] Interview Questions
- Azure Data Lake For Beginners: All you Need To Know
- Batch Processing Vs Stream Processing: All you Need To Know
- Reading and Writing Data In DataBricks
Next Task For You
In our Azure Data Engineer training program, we will cover all the exam objectives, 27 Hands-On Labs, and practice tests. If you want to begin your journey towards becoming a Microsoft Certified: Azure Data Engineer Associate check our FREE CLASS.
The post Azure Synapse Link | Hybrid Transactional Analytical Processing appeared first on Cloud Training Program.