Azure Data Factory | Lab: Copy Data From Blob Storage To SQL Database | K21 Academy

Azure Data Factory is defined as a cloud-based ETL and data integration service. The aim of Azure Data Factory is to fetch data from one or more data sources and load them into a format that we process.

The data sources might contain noise that we need to filter out. Azure Data Factory enables us to pull the interesting data and remove the rest.
Azure Data Factory to ingest data and load the data from a variety of sources into a variety of destinations i.e. Azure data lake. It can create data-driven pipelines for orchestrating data movement and transforming data at scale.

➢ Azure Blob Storage
Azure Blob storage is Microsoft’s Azure object storage solution for the cloud. It is designed for optimized and storing massive amounts of unstructured data.

➢ Azure Blob storage offers three types of resources:
1. The storage account
2. A container in the storage account
3. A blob in a container

➢ Azure SQL Database
It is a fully-managed platform as a service. Here the platform manages aspects such as database software upgrades, patching, backups, the monitoring.

➢ Types of Deployment Options for the SQL Database:
1. Single Database
2. Elastics Pool
3. Managed Instance

To know more, check this video from K21academy
Where we explain:
00:16 = Introduction to Azure Data Factory
00:31 = Agenda
02:08 = Overview of Azure Data Factory
02:54 = How to Create Storage Account
04:42 = Create SQL Database
06:23 = Create Data factory
10:17 = How to copy Activity with Azure Data Factory
10:53 = Overview of Azure Blob Storage
16:03 = FREE Training
16:12 = Learning Path for Azure Data Engineer Certification [DP-200 & DP-201]
16:24 = Registration link for FREE Class

#datafactory #azuredatafactory #microsoftazure #azuredatafactorytutorialforbeginners #understandingazuredatafactory #introductiontoazuredatafactory #azuredatafactorytutorial #azuredatafactorytraining #microsoftazuredatafactory #azure #k21academy #askatul

----------------------------------------------------------------------------------------------------------------------
Subscribe To K21Academys YouTube Channels for more FREE Information on:

---------------------------------------------------------------------------------------------------------------
LET'S CONNECT ON SOCIAL ↴

[ashutoshnayak]

While creating azure data factory is it to keep public end point or private end point network?

[anamikasar]

sir can we make it dynamically, suppose in a blob storage there are so many files in different folders and want to load all of those files present in different folders from blob storage to sql db then how we can create to featch those data/records dynamically and dump it to the sql server dynamically ? please help sir

[raviteja_asr]

Hi..how to add multiple modified files from blob to SQL as we cant create new tables in SQL at once..

[denwo]

Why would you not just store the data straight into Azure SQL Database tables? And bypass Azure Blob Storage?

[saumyajitparida]

You stored the the last column, which was integer, as string. This is completely misleading. You should have shown in the video, the process of maintaining the same data format (rather saving column3 as string in SQL)