How to Load Multiple CSV Files to Multiple Tables According to File Name in Azure Data Factory

How to Load Multiple .csv files (each file with a different schema) from blob to their respective tables in azure SQL server in ADF. In this video, we are going to learn how to create a new table in ADF deepening upon a list of columns in CSV files. Each file will be loaded to a separate table according to FileName. If the table already exists, then it will be loaded without dropping and recreating. IF a table does not exist then it will create a new table and then load the data from CSV File.
How to Load Multiple CSV Files to Multiple Tables According to FileName in Azure Data Factory,
How to Load Multiple .csv files (each file with different schema) from blob to their respective tables in azure SQL server in ADF,
How to load multiple CSV files from Azure Blob to Azure SQL Database,
Load files dynamically in Azure Data Factory,
How to map CSV files dynamically to Tables in Azure Data Factory,
Azure Data Factory Tutorial 2021,TechbrothersIT,ADF Tutorial for beginners
#AzureDataFactoryTutorial

[mairaacevedo]

Fantastic content. I combined some of your videos to do a bit of a complex task and I'm so happy it worked!. Thanks heaps!

[jayong]

Thank you! This is exactly what I was trying to configure.

[bitips]

Thanks for share this knowledge. It is fantastic !

[neftalimich]

Thank you very much, was really helpful.

[piraviperumal]

Hi Brother,

Not sure whether Azure team fixed it or not but @replace(item().name, '.txt', '') is working fine. I guess you have missed @ sign before replace function in your attempt.

[vishal-xfev]

Hi Brother,

Great Video & thanks for sharing :-)

[purushothamnaidu]

Sir...Can you show once how to load the files available in blob container and load into multiple existing tables in azure sql database, that would be really helpful to me

[TheMadPiggy]

works like a charm, however the auto created tables all are nvarchar(MAX). Not the best for database size, not for useability. Any way around this.

[tomasoon]

very great tutorial, i have a question, if I run a pipeline, and there's a new csv file in the bucket with the same schema as other, this method will apend the data to the table with same schema or will create another one?

[devops]

Hi Sir,
I am able to insert the data using dynamic CSV files, Could you please help me in upserting the data ?

[williamtenhoven]

Hi, thanks for this ! 1 question. suppose I wanted to convert the csv files to parquet files the how would I proceed ? I used the concat replace, but looking at the target parquet files they seem to be corrupted : The file 'Emp1' may not render correctly as it contains an unrecognized extension. @concat(replace(item().name, 'csv', 'parquet')) does not work Any suggestions ? Thanks

[ayushikankane]

If csv file is hqving some columna as json structure than how to proceed?

[boatengappiah]

Great videos. I however don't see any video on SharePoint with ADF. Do you have a video or can you make one? Thank you

[insane]

Small query sir, once created the table, again if new data or new files come with suffix changes like date change then again it create new table or insert the data into the already created table coz you are using auto created option . Thank you in advance

[uditbhargava]

Sir can we use split() function to remove .txt ?

[Eraldi]

Hi TechBrothers, thanks for this very useful video.
I had a question, I am trying to truncate the tables with the following
@{concat('truncate table', item().name)} but is not working for me, giving an error
Please advise.
Thank You

[kiranreddy]

HI, if file names are like emp1, emp2, emp3 etc. in this case how we can write a expression to remove numb
ers in REPLACE. could you help us.

[viswaanand]

Hi
I can see my csv files in SSMS but cannot see in table format in SSMS also it is in CSV format did i miss anything?

[Deezubwun]

Hi. This was a great help to me. One issue I am having is the data is failing to load due to multiple data type errors (such as String to DATETIME). As the data in the CSV is exported as string, do you have a way of mapping the formatting of each field which is a problem, bearing in mind the columns may be named something different?

[rohitsethi]

hi im rohit can we use copy data activity from CSV files if not why ?