20200723 - Reading and writing data from/to SQL Server using Apache Spark

This video shows you how to read and write data from/to SQL Server using Apache Spark

Prerequisite:
1. A SQL Server Instance with 2 databases
2. Local Apache Spark env configured
3. Visual Studio Code (VS Code) with Pyspark installed and configured

Steps:
2: Copy this jar to jars folder under SPARK_HOME

[CodePractice-nlwt]

The JAR is not there in the github link that you provided.

[harshitagrawal]

I want to copy big tables, read as well as write in batches. Could you help me out.

[JL-qcgq]

How can we do Merge (Insert new rows from the source, update modified rows from the source and update to inactive the deleted rows from the source) in dataframe writer?

[prashantmathur]

Please explain how to configure VSCode for PySpark. I want intellisense but not able to get

[rhotimie]

Apache 3.1.2 is throwing using your approach. could you please help