Writing Data to Azure SQL Data Warehouse with Azure Databricks

preview_player
Показать описание
Welcome to the Month of Azure Databricks presented by Advancing Analytics.

In this video Simon takes you though how write data from a Spark DataFrame to Azure SQL Data Warehouse. While this may sound simple, to get the most from Azure SQL Data Warehouse, you need to be using PolyBase to read in data.

Starting your first project with Azure Databricks? Well get in touch!

You might be also interested in our other videos and blogs:

If you like the videos but want to go deeper, then take a look at our Azure Databricks training course:
Рекомендации по теме
Комментарии
Автор

Informative appreciated
Kindly upload more videos on Databricks

suryapratap
Автор

We can do read and write to azure sql table and other tables in databricks via JDBC. But how can we do update or delete on the same tables?

JL-qcgq
Автор

Great content! one questions: does this also apply to a Azure SQL DB as a destination?

MrRegaschardijn
Автор

how can i use Azure Databricks Delta Lake connector within a pipeline to pull from Databricks to Synapse?

seaTiger
Автор

Very nice explanation!
I have implemented the same with ref to this great video. In my case in mode i am passing "append" but still this code is distorting the table structure. Example :- I have underlying database table in synapse TableA with 4 columns A*, B*, C, D. Please note that here * represents NULL not allow for columns A and B. Now in my PySpark dataframe I have 2 mandatory columns dfTableA (A, B). Now the problem is that when I am doing bulk insert with approach even after mentioning the mode as "append" it is overwriting the entire  and changing it's structure from TableA (A*, B*, C, D) -- >  TableA (A*, B*)

varunthakur
Автор

I just tested this in the dedicated SQL pool and it appears to create a regular table there and not an external table as stated in the video.

Also, the files that are put in the tempDir appears to be in Parquet-format if anyone wondered. :)

Marruisz
Автор

This kind of video should contain more hands on....NOT that much helpful....sorry

indrajit
welcome to shbcf.ru